Service Mesh

Configuration affecting the service mesh as a whole.

AuthenticationPolicy

AuthenticationPolicy defines authentication policy. It can be set for different scopes (mesh, service …), and the most narrow scope with non-INHERIT value will be used. Mesh policy cannot be INHERIT.

NameDescription
NONE

Do not encrypt Envoy to Envoy traffic.

MUTUAL_TLS

Envoy to Envoy traffic is wrapped into mutual TLS connections.

INHERIT

Use the policy defined by the parent scope. Should not be used for mesh policy.

ConfigSource

ConfigSource describes information about a configuration store inside a mesh. A single control plane instance can interact with one or more data sources.

FieldTypeDescription
addressstring

Address of the server implementing the Istio Mesh Configuration protocol (MCP). Can be IP address or a fully qualified DNS name. Use fs:/// to specify a file-based backend with absolute path to the directory.

tlsSettingsistio.networking.v1alpha3.TLSSettings

Use the tls_settings to specify the tls mode to use. If the MCP server uses Istio mutual TLS and shares the root CA with Pilot, specify the TLS mode as ISTIO_MUTUAL.

LocalityLoadBalancerSetting

Locality-weighted load balancing allows administrators to control the distribution of traffic to endpoints based on the localities of where the traffic originates and where it will terminate. These localities are specified using arbitrary labels that designate a hierarchy of localities in {region}/{zone}/{sub-zone} form. For additional detail refer to Locality Weight The following example shows how to setup locality weights mesh-wide.

Given a mesh with workloads and their service deployed to “us-west/zone1/” and “us-west/zone2/”. This example specifies that when traffic accessing a service originates from workloads in “us-west/zone1/”, 80% of the traffic will be sent to endpoints in “us-west/zone1/”, i.e the same zone, and the remaining 20% will go to endpoints in “us-west/zone2/”. This setup is intended to favor routing traffic to endpoints in the same locality. A similar setting is specified for traffic originating in “us-west/zone2/”.

  distribute:
    - from: us-west/zone1/*
      to:
        "us-west/zone1/*": 80
        "us-west/zone2/*": 20
    - from: us-west/zone2/*
      to:
        "us-west/zone1/*": 20
        "us-west/zone2/*": 80

If the goal of the operator is not to distribute load across zones and regions but rather to restrict the regionality of failover to meet other operational requirements an operator can set a ‘failover’ policy instead of a ‘distribute’ policy.

The following example sets up a locality failover policy for regions. Assume a service resides in zones within us-east, us-west & eu-west this example specifies that when endpoints within us-east become unhealthy traffic should failover to endpoints in any zone or sub-zone within eu-west and similarly us-west should failover to us-east.

 failover:
   - from: us-east
     to: eu-west
   - from: us-west
     to: us-east

Locality load balancing settings.

FieldTypeDescription
distributeLocalityLoadBalancerSetting.Distribute[]

Optional: only one of distribute or failover can be set. Explicitly specify loadbalancing weight across different zones and geographical locations. Refer to Locality weighted load balancing If empty, the locality weight is set according to the endpoints number within it.

failoverLocalityLoadBalancerSetting.Failover[]

Optional: only failover or distribute can be set. Explicitly specify the region traffic will land on when endpoints in local region becomes unhealthy. Should be used together with OutlierDetection to detect unhealthy endpoints. Note: if no OutlierDetection specified, this will not take effect.

LocalityLoadBalancerSetting.Distribute

Describes how traffic originating in the ‘from’ zone or sub-zone is distributed over a set of ‘to’ zones. Syntax for specifying a zone is {region}/{zone}/{sub-zone} and terminal wildcards are allowed on any segment of the specification. Examples: * - matches all localities us-west/* - all zones and sub-zones within the us-west region us-west/zone-1/* - all sub-zones within us-west/zone-1

FieldTypeDescription
fromstring

Originating locality, ‘/’ separated, e.g. ‘region/zone/sub_zone’.

tomap<string, uint32>

Map of upstream localities to traffic distribution weights. The sum of all weights should be == 100. Any locality not assigned a weight will receive no traffic.

LocalityLoadBalancerSetting.Failover

Specify the traffic failover policy across regions. Since zone and sub-zone failover is supported by default this only needs to be specified for regions when the operator needs to constrain traffic failover so that the default behavior of failing over to any endpoint globally does not apply. This is useful when failing over traffic across regions would not improve service health or may need to be restricted for other reasons like regulatory controls.

FieldTypeDescription
fromstring

Originating region.

tostring

Destination region the traffic will fail over to when endpoints in the ‘from’ region becomes unhealthy.

MeshConfig

MeshConfig defines mesh-wide variables shared by all Envoy instances in the Istio service mesh.

NOTE: This configuration type should be used for the low-level global configuration, such as component addresses and port numbers. It should not be used for the features of the mesh that can be scoped by service or by namespace. Some of the fields in the mesh config are going to be deprecated and replaced with several individual configuration types (for example, tracing configuration).

FieldTypeDescription
mixerCheckServerstring

Address of the server that will be used by the proxies for policy check calls. By using different names for mixerCheckServer and mixerReportServer, it is possible to have one set of Mixer servers handle policy check calls while another set of Mixer servers handle telemetry calls.

NOTE: Omitting mixerCheckServer while specifying mixerReportServer is equivalent to setting disablePolicyChecks to true.

mixerReportServerstring

Address of the server that will be used by the proxies for policy report calls.

disablePolicyChecksbool

Disable policy checks by the Mixer service. Default is false, i.e. Mixer policy check is enabled by default.

policyCheckFailOpenbool

Allow all traffic in cases when the Mixer policy service cannot be reached. Default is false which means the traffic is denied when the client is unable to connect to Mixer.

sidecarToTelemetrySessionAffinitybool

Enable session affinity for Envoy Mixer reports so that calls from a proxy will always target the same Mixer instance.

proxyListenPortint32

Port on which Envoy should listen for incoming connections from other services.

proxyHttpPortint32

Port on which Envoy should listen for HTTP PROXY requests if set.

connectTimeoutgoogle.protobuf.Duration

Connection timeout used by Envoy. (MUST BE >=1ms)

tcpKeepaliveistio.networking.v1alpha3.ConnectionPoolSettings.TCPSettings.TcpKeepalive

If set then set SO_KEEPALIVE on the socket to enable TCP Keepalives.

ingressClassstring

Class of ingress resources to be processed by Istio ingress controller. This corresponds to the value of “kubernetes.io/ingress.class” annotation.

ingressServicestring

Name of theKubernetes service used for the istio ingress controller.

ingressControllerModeMeshConfig.IngressControllerMode

Defines whether to use Istio ingress controller for annotated or all ingress resources.

enableTracingbool

Flag to control generation of trace spans and request IDs. Requires a trace span collector defined in the proxy configuration.

accessLogFilestring

File address for the proxy access log (e.g. /dev/stdout). Empty value disables access logging.

accessLogFormatstring

Format for the proxy access log Empty value results in proxy’s default access log format

accessLogEncodingMeshConfig.AccessLogEncoding

Encoding for the proxy access log (text or json). Default value is text.

defaultConfigProxyConfig

Default proxy config used by the proxy injection mechanism operating in the mesh (e.g. Kubernetes admission controller) In case of Kubernetes, the proxy config is applied once during the injection process, and remain constant for the duration of the pod. The rest of the mesh config can be changed at runtime and config gets distributed dynamically.

outboundTrafficPolicyMeshConfig.OutboundTrafficPolicy

Set the default behavior of the sidecar for handling outbound traffic from the application. If your application uses one or more external services that are not known apriori, setting the policy to ALLOWANY will cause the sidecars to route any unknown traffic originating from the application to its requested destination. Users are strongly encouraged to use ServiceEntries to explicitly declare any external dependencies, instead of using allowany, so that traffic to these services can be monitored.

enableClientSidePolicyCheckbool

Enables clide side policy checks.

sdsUdsPathstring

Unix Domain Socket through which Envoy communicates with NodeAgent SDS to get key/cert for mTLS. Use secret-mount files instead of SDS if set to empty.

configSourcesConfigSource[]

ConfigSource describes a source of configuration data for networking rules, and other Istio configuration artifacts. Multiple data sources can be configured for a single control plane.

trustDomainstring

The trust domain corresponds to the trust root of a system. Refer to SPIFEE-ID Fallback to old identity format(without trust domain) if not set.

localityLbSettingLocalityLoadBalancerSetting

Locality based load balancing distribution or failover settings.

dnsRefreshRategoogle.protobuf.Duration

Configures DNS refresh rate for Envoy clusters of type STRICT_DNS

MeshConfig.AccessLogEncoding

NameDescription
TEXT
JSON

MeshConfig.IngressControllerMode

NameDescription
OFF

Disables Istio ingress controller.

DEFAULT

Istio ingress controller will act on ingress resources that do not contain any annotation or whose annotations match the value specified in the ingress_class parameter described earlier. Use this mode if Istio ingress controller will be the default ingress controller for the entireKubernetes cluster.

STRICT

Istio ingress controller will only act on ingress resources whose annotations match the value specified in the ingress_class parameter described earlier. Use this mode if Istio ingress controller will be a secondary ingress controller (e.g., in addition to a cloud-provided ingress controller).

MeshConfig.OutboundTrafficPolicy

FieldTypeDescription
modeMeshConfig.OutboundTrafficPolicy.Mode

MeshConfig.OutboundTrafficPolicy.Mode

NameDescription
REGISTRY_ONLY

outbound traffic will be restricted to services defined in the service registry as well as those defined through ServiceEntries

ALLOW_ANY

outbound traffic to unknown destinations will be allowed, in case there are no services or ServiceEntries for the destination port

MeshNetworks

MeshNetworks (config map) provides information about the set of networks inside a mesh and how to route to endpoints in each network. For example

MeshNetworks(file/config map): networks: network1: - endpoints: - fromRegistry: registry1 #must match secret name in Kubernetes - fromCidr: 192.168.100.0/22 #a VM network for example gateways: - registryServiceName: istio-ingressgateway.istio-system.svc.cluster.local port: 15443 locality: us-east-1a - address: 192.168.100.1 port: 15443 locality: us-east-1a

FieldTypeDescription
networksmap<string, Network>

REQUIRED: The set of networks inside this mesh. Each network should have a unique name and information about how to infer the endpoints in the network as well as the gateways associated with the network.

Network

Network provides information about the endpoints in a routable L3 network. A single routable L3 network can have one or more service registries. Note that the network has no relation to the locality of the endpoint. The endpoint locality will be obtained from the service registry.

FieldTypeDescription
endpointsNetwork.NetworkEndpoints[]

REQUIRED: The list of endpoints in the network (obtained through the constituent service registries or from CIDR ranges). All endpoints in the network are directly accessible to one another.

gatewaysNetwork.IstioNetworkGateway[]

REQUIRED: Set of gateways associated with the network.

Network.IstioNetworkGateway

The gateway associated with this network. Traffic from remote networks will arrive at the specified gateway:port. All incoming traffic must use mTLS.

FieldTypeDescription
registryServiceNamestring (oneof)

A fully qualified domain name of the gateway service. Pilot will lookup the service from the service registries in the network and obtain the endpoint IPs of the gateway from the service registry. Note that while the service name is a fully qualified domain name, it need not be resolvable outside the orchestration platform for the registry. e.g., this could be istio-ingressgateway.istio-system.svc.cluster.local.

addressstring (oneof)

IP address or externally resolvable DNS address associated with the gateway.

portuint32

REQUIRED: The port associated with the gateway.

localitystring

The locality associated with an explicitly specified gateway (i.e. ip)

Network.NetworkEndpoints

NetworkEndpoints describes how the network associated with an endpoint should be inferred. An endpoint will be assigned to a network based on the following rules:

  1. Implicitly: If the registry explicitly provides information about the network to which the endpoint belongs to. In some cases, its possible to indicate the network associated with the endpoint by adding the ISTIO_META_NETWORK environment variable to the sidecar.

  2. Explicitly:

a. By matching the registry name with one of the “fromRegistry” in the mesh config. A “from_registry” can only be assigned to a single network.

b. By matching the IP against one of the CIDR ranges in a mesh config network. The CIDR ranges must not overlap and be assigned to a single network.

(2) will override (1) if both are present.

FieldTypeDescription
fromCidrstring (oneof)

A CIDR range for the set of endpoints in this network. The CIDR ranges for endpoints from different networks must not overlap.

fromRegistrystring (oneof)

Add all endpoints from the specified registry into this network. The names of the registries should correspond to the secret name that was used to configure the registry (Kubernetes multicluster) or supplied by MCP server.

ProxyConfig

ProxyConfig defines variables for individual Envoy instances.

FieldTypeDescription
configPathstring

Path to the generated configuration file directory. Proxy agent generates the actual configuration and stores it in this directory.

binaryPathstring

Path to the proxy binary

serviceClusterstring

Service cluster defines the name for the service_cluster that is shared by all Envoy instances. This setting corresponds to –service-cluster flag in Envoy. In a typical Envoy deployment, the service-cluster flag is used to identify the caller, for source-based routing scenarios.

Since Istio does not assign a local service/service version to each Envoy instance, the name is same for all of them. However, the source/caller’s identity (e.g., IP address) is encoded in the –service-node flag when launching Envoy. When the RDS service receives API calls from Envoy, it uses the value of the service-node flag to compute routes that are relative to the service instances located at that IP address.

drainDurationgoogle.protobuf.Duration

The time in seconds that Envoy will drain connections during a hot restart. MUST be >=1s (e.g., 1s/1m/1h)

parentShutdownDurationgoogle.protobuf.Duration

The time in seconds that Envoy will wait before shutting down the parent process during a hot restart. MUST be >=1s (e.g., 1s/1m/1h). MUST BE greater than drainduration_ parameter.

discoveryAddressstring

Address of the discovery service exposing xDS with mTLS connection.

zipkinAddressstring

Address of the Zipkin service (e.g. zipkin:9411). DEPRECATED: Use tracing instead.

connectTimeoutgoogle.protobuf.Duration

Connection timeout used by Envoy for supporting services. (MUST BE >=1ms)

statsdUdpAddressstring

IP Address and Port of a statsd UDP listener (e.g. 10.75.241.127:9125).

envoyMetricsServiceAddressstring

Address of the Envoy Metrics Service implementation (e.g. metrics-service:15000). See Metric Service for details about Envoy’s Metrics Service API.

proxyAdminPortint32

Port on which Envoy should listen for administrative commands.

controlPlaneAuthPolicyAuthenticationPolicy

Authentication policy defines the global switch to control authentication for Envoy-to-Envoy communication for istio components Mixer and Pilot.

customConfigFilestring

File path of custom proxy configuration, currently used by proxies in front of Mixer and Pilot.

statNameLengthint32

Maximum length of name field in Envoy’s metrics. The length of the name field is determined by the length of a name field in a service and the set of labels that comprise a particular version of the service. The default value is set to 189 characters. Envoy’s internal metrics take up 67 characters, for a total of 256 character name per metric. Increase the value of this field if you find that the metrics from Envoys are truncated.

concurrencyint32

The number of worker threads to run. Default value is number of cores on the machine.

proxyBootstrapTemplatePathstring

Path to the proxy bootstrap template file

interceptionModeProxyConfig.InboundInterceptionMode

The mode used to redirect inbound traffic to Envoy.

tracingTracing

Tracing configuration to be used by the proxy.

ProxyConfig.InboundInterceptionMode

The mode used to redirect inbound traffic to Envoy. This setting has no effect on outbound traffic: iptables REDIRECT is always used for outbound connections.

NameDescription
REDIRECT

The REDIRECT mode uses iptables REDIRECT to NAT and redirect to Envoy. This mode loses source IP addresses during redirection.

TPROXY

The TPROXY mode uses iptables TPROXY to redirect to Envoy. This mode preserves both the source and destination IP addresses and ports, so that they can be used for advanced filtering and manipulation. This mode also configures the sidecar to run with the CAPNETADMIN capability, which is required to use TPROXY.

Tracing

Tracing defines configuration for the tracing performed by Envoy instances.

FieldTypeDescription
zipkinTracing.Zipkin (oneof)

Use a Zipkin tracer.

lightstepTracing.Lightstep (oneof)

Use a LightStep tracer.

datadogTracing.Datadog (oneof)

Use a Datadog tracer.

Tracing.Datadog

Datadog defines configuration for a Datadog tracer.

FieldTypeDescription
addressstring

Address of the Datadog Agent.

Tracing.Lightstep

Lightstep defines configuration for a LightStep tracer.

FieldTypeDescription
addressstring

Address of the LightStep Satellite pool.

accessTokenstring

The LightStep access token.

securebool

True if a secure connection should be used when communicating with the pool.

cacertPathstring

Path to the trusted cacert used to authenticate the pool.

Tracing.Zipkin

Zipkin defines configuration for a Zipkin tracer.

FieldTypeDescription
addressstring

Address of the Zipkin service (e.g. zipkin:9411).