Configure tracing using MeshConfig and Pod annotations
Istio provides the ability to configure advanced tracing options, such as sampling rate and adding custom tags to reported spans. Sampling is a beta feature, but adding custom tags and tracing tag length are considered in-development for this release.
Before you begin
Ensure that your applications propagate tracing headers as described here7.
Follow the tracing installation guide located under Integrations8 based on your preferred tracing backend to install the appropriate addon and configure your Istio proxies to send traces to the tracing deployment.
Available tracing configurations
You can configure the following tracing options in Istio:
Random sampling rate for percentage of requests that will be selected for trace generation.
Maximum length of the request path after which the path will be truncated for reporting. This can be useful in limiting trace data storage specially if you’re collecting traces at ingress gateways.
Adding custom tags in spans. These tags can be added based on static literal values, environment values or fields from request headers. This can be used to inject additional information in spans specific to your environment.
There are two ways you can configure tracing options:
Globally via
MeshConfig
options.Per-pod annotations for workload specific customization.
Installation
Using these features opens new possibilities for managing traces in your environment.
In this example, we will sample all traces and add a tag named clusterID
using the ISTIO_META_CLUSTER_ID
environment variable injected into your pod. Only the
first 256 characters of the value will be used.
Using MeshConfig
for trace settings
All tracing options can be configured globally via MeshConfig
.
To simplify configuration, it is recommended to create a single YAML file
which you can pass to the istioctl install -f
command.
Using proxy.istio.io/config
annotation for trace settings
You can add the proxy.istio.io/config
annotation to your Pod metadata
specification to override any mesh-wide tracing settings.
For instance, to modify the sleep
deployment shipped with Istio you would add
the following to samples/sleep/sleep.yaml
:
Customizing Trace sampling
The sampling rate option can be used to control what percentage of requests get reported to your tracing system. This should be configured depending upon your traffic in the mesh and the amount of tracing data you want to collect. The default rate is 1%.
To modify the default random sampling to 50, add the following option to your
tracing.yaml
file.
The sampling rate should be in the range of 0.0 to 100.0 with a precision of 0.01. For example, to trace 5 requests out of every 10000, use 0.05 as the value here.
Customizing tracing tags
Custom tags can be added to spans based on literals, environmental variables and client request headers in order to provide additional information in spans specific to your environment.
You can customize the tags using any of the three supported options below.
Literal represents a static value that gets added to each span.
Environmental variables can be used where the value of the custom tag is populated from a workload proxy environment variable.
Client request header option can be used to populate tag value from an incoming client request header.
Customizing tracing tag length
By default, the maximum length for the request path included as part of the HttpUrl
span tag is 256.
To modify this maximum length, add the following to your tracing.yaml
file.