Metrics and Logs FAQ

You can collect telemetry data about Istio using Prometheus. And then use Prometheus’s HTTP API to query that data.

It is sometimes useful to exclude access to specific URLs from being reported. For example, you might wish to exclude some health-checking URLs. You can skip telemetry reports specific to certain URLs by excluding them using a match clause of the telemetry configuration such as:

match: source.name != "health"

You can use docker-compose to install Prometheus, similar to when you install your application. Also, without the Kubernetes API server, components such as Mixer will need local configuration for rules/handlers/instances.

You can enable tracing to determine the flow of a request in Istio.

Additionally, you can use the following commands to know more about the state of the mesh:

• istioctl proxy-config: Retrieve information about proxy configuration when running in Kubernetes:

  # Retrieve information about bootstrap configuration for the Envoy instance in the specified pod.
  $ istioctl proxy-config bootstrap productpage-v1-bb8d5cbc7-k7qbm
  
  # Retrieve information about cluster configuration for the Envoy instance in the specified pod.
  $ istioctl proxy-config cluster productpage-v1-bb8d5cbc7-k7qbm
  
  # Retrieve information about listener configuration for the Envoy instance in the specified pod.
  $ istioctl proxy-config listener productpage-v1-bb8d5cbc7-k7qbm
  
  # Retrieve information about route configuration for the Envoy instance in the specified pod.
  $ istioctl proxy-config route productpage-v1-bb8d5cbc7-k7qbm
  
  # Retrieve information about endpoint configuration for the Envoy instance in the specified pod.
  $ istioctl proxy-config endpoints productpage-v1-bb8d5cbc7-k7qbm
  
  # Try the following to discover more proxy-config commands
  $ istioctl proxy-config --help
  

• kubectl get: Gets information about different resources in the mesh along with routing configuration:

  # List all virtual services
  $ kubectl get virtualservices
  

• Mixer access logs: Mixer writes access logs that contain information about requests. You can get them with:

  # Fill <istio namespace> with the namespace of your istio mesh. Ex: istio-system
  $ TELEMETRY_POD=`kubectl get po -n <istio namespace> | grep istio-telemetry | awk '{print $1;}'`
  $ kubectl logs $TELEMETRY_POD -c mixer  -n istio-system  | grep accesslog
  

Yes. Istio ships with configuration for Prometheus that enables collection of application metrics when mutual TLS is enabled or disabled.

The kubernetes-pods job collects application metrics from pods in environments without mutual TLS. The kubernetes-pods-istio-secure job collects metrics from application pods when mutual TLS is enabled for Istio.

Both jobs require that the following annotations are added to any deployments from which application metric collection is desired:

• prometheus.io/scrape: "true"
• prometheus.io/path: "<metrics path>"
• prometheus.io/port: "<metrics port>"

A few notes:

• If the Prometheus pod started before the Istio Citadel pod could generate the required certificates and distribute them to Prometheus, the Prometheus pod will need to be restarted in order to collect from mutual TLS-protected targets.
• If your application exposes Prometheus metrics on a dedicated port, that port should be added to the service and deployment specifications.