Debugging Virtual Machines
This page describes how to troubleshoot issues with Istio deployed to Virtual Machines. Before reading this, you should take the steps in Virtual Machine Installation4. Additionally, Virtual Machine Architecture5 can help you understand how the components interact.
Troubleshooting an Istio Virtual Machine installation is similar to troubleshooting issues with proxies running inside Kubernetes, but there are some key differences to be aware of.
While much of the same information is available on both platforms, accessing this information differs.
Monitoring health
The Istio sidecar is typically run as a systemd
unit. To ensure its running properly, you can check that status:
Additionally, the sidecar health can be programmatically check at its health endpoint:
Logs
Logs for the Istio proxy can be found in a few places.
To access the systemd
logs, which has details about the initialization of the proxy:
The proxy will redirect stderr
and stdout
to /var/log/istio/istio.err.log
and /var/log/istio/istio.log
, respectively.
To view these in a format similar to kubectl
:
Log levels can be modified by changing the cluster.env
configuration file. Make sure to restart istio
if it is already running:
Iptables
To ensure iptables
rules have been successfully applied:
Istioctl
Most istioctl
commands will function properly with virtual machines. For example, istioctl proxy-status
can be used to view all connected proxies:
However, istioctl proxy-config
relies on functionality in Kubernetes to connect to a proxy, which will not work for virtual machines.
Instead, a file containing the configuration dump from Envoy can be passed. For example:
Automatic registration
When a virtual machine connects to Istiod, a WorkloadEntry
will automatically be created. This enables
the virtual machine to become a part of a Service
, similar to an Endpoint
in Kubernetes.
To check these are created correctly:
Certificates
Virtual machines handle certificates differently than Kubernetes Pods, which use a Kubernetes-provided service account token to authenticate and renew mTLS certificates. Instead, existing mTLS credentials are used to authenticate with the certificate authority and renew certificates.
The status of these certificates can be viewed in the same way as in Kubernetes:
Additionally, these are persisted to disk to ensure downtime or restarts do not lose state.