Install Istio with the Istio CNI plugin

Follow this guide to install, configure, and use an Istio mesh using the Istio Container Network Interface (CNI) plugin.

By default Istio injects an initContainer, istio-init, in pods deployed in the mesh. The istio-init container sets up the pod network traffic redirection to/from the Istio sidecar proxy. This requires the user or service-account deploying pods to the mesh to have sufficient Kubernetes RBAC permissions to deploy containers with the NET_ADMIN and NET_RAW capabilities. Requiring Istio users to have elevated Kubernetes RBAC permissions is problematic for some organizations’ security compliance. The Istio CNI plugin is a replacement for the istio-init container that performs the same networking functionality but without requiring Istio users to enable elevated Kubernetes RBAC permissions.

The Istio CNI plugin performs the Istio mesh pod traffic redirection in the Kubernetes pod lifecycle’s network setup phase, thereby removing the requirement for the NET_ADMIN and NET_RAW capabilities for users deploying pods into the Istio mesh. The Istio CNI plugin replaces the functionality provided by the istio-init container.

Prerequisites

1. Install Kubernetes with the container runtime supporting CNI and kubelet configured with the main CNI plugin enabled via --network-plugin=cni.

   • AWS EKS, Azure AKS, and IBM Cloud IKS clusters have this capability.
   • Google Cloud GKE clusters require that the network-policy feature is enabled to have Kubernetes configured with network-plugin=cni.
   • OpenShift has CNI enabled by default.

2. Install Kubernetes with the ServiceAccount admission controller enabled.

   • The Kubernetes documentation highly recommends this for all Kubernetes installations where ServiceAccounts are utilized.

Basic Installation

In most environments, a basic Istio cluster with CNI enabled can be installed using the following command:

$ cat <<EOF > istio-cni.yaml
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  components:
    cni:
      enabled: true
  values:
    cni:
      excludeNamespaces:
       - istio-system
       - kube-system
      logLevel: info
EOF
$ istioctl install -f istio-cni.yaml

Advanced Installation

1. Determine the Kubernetes environment’s CNI plugin --cni-bin-dir and --cni-conf-dir settings. Refer to Hosted Kubernetes settings for any non-default settings required.

2. Install Istio CNI and Istio using istioctl. Refer to the Istio install instructions and pass --set components.cni.enabled=true option. Pass --set values.cni.cniBinDir=... and/or --set values.cni.cniConfDir=... options when installing istio-cni if non-default, as determined in the previous step.

Helm chart parameters

The following table shows all the options that the istio-cni configuration supports:

These options are accessed through values.cni.<option-name> in istioctl manifest commands, either as a --set flag, or the corresponding path in a custom overlay file.

Excluding specific Kubernetes namespaces

This example uses istioctl to perform the following tasks:

• Install the Istio CNI plugin.
• Configure its log level.
• Ignore the pods in the following namespaces:
  • istio-system
  • foo_ns
  • bar_ns

Refer to the Customizable Install with Istioctl for complete instructions.

Use the following command to render and apply Istio CNI components and override the default configuration of the logLevel and excludeNamespaces parameters for istio-cni:

Create a IstioOperator CR yaml locally with your override to install istio, e.g. cni.yaml

apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  components:
    cni:
      enabled: true
  values:
    cni:
      excludeNamespaces:
       - istio-system
       - kube-system
       - foo_ns
       - bar_ns
      logLevel: info

$ istioctl install -f cni.yaml

Hosted Kubernetes settings

The Istio CNI solution is not ubiquitous. Some platforms, especially hosted Kubernetes environments, do not enable the CNI plugin in the kubelet configuration. The istio-cni plugin is expected to work with any hosted Kubernetes leveraging CNI plugins. The following table shows the required settings for many common Kubernetes environments.

Instructions for Istio 1.4.x and OpenShift

Due to a limitation in istioctl 1.4.x using --set with escaped strings, a YAML file is necessary to set values.sidecarInjectorWebhook.injectedAnnotations to install Istio on OpenShift. Create the YAML file:

cat <<'EOF' > cni-annotations.yaml
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  components:
    cni:
      enabled: true
      namespace: kube-system
  values:
    cni:
      chained: false
      cniBinDir: /var/lib/cni/bin
      cniConfDir: /etc/cni/multus/net.d
      cniConfFileName: istio-cni.conf
    sidecarInjectorWebhook:
      injectedAnnotations:
        "k8s.v1.cni.cncf.io/networks": istio-cni
EOF

Then pass this file as an argument to istioctl, for example:

$ istioctl install -f cni-annotations.yaml

You can pass other command line arguments with --set if necessary.

GKE setup

1. Refer to the procedure to prepare a GKE cluster for Istio and enable network-policy in your cluster.

2. Install Istio CNI via Istioctl including the --set values.cni.cniBinDir=/home/kubernetes/bin option. For example, the following istioctl manifest command sets the values.cni.cniBinDir value for a GKE cluster:

   $ istioctl install --set values.cni.cniBinDir=/home/kubernetes/bin \
       --set components.cni.enabled=true \
       --set components.cni.namespace=kube-system
   

Sidecar injection compatibility

The use of the Istio CNI plugin requires Kubernetes pods to be deployed with a sidecar injection method that uses the istio-sidecar-injector configmap created from the installation with the --set cni.enabled=true option. Refer to Istio sidecar injection for details about Istio sidecar injection methods.

The following sidecar injection methods are supported for use with the Istio CNI plugin:

1. Automatic sidecar injection

2. Manual sidecar injection with the istio-sidecar-injector configmap

   1. istioctl kube-inject using the configmap directly:

      $ istioctl kube-inject -f deployment.yaml -o deployment-injected.yaml --injectConfigMapName istio-sidecar-injector
      $ kubectl apply -f deployment-injected.yaml
      

   2. istioctl kube-inject using a file created from the configmap:

      $ kubectl -n istio-system get configmap istio-sidecar-injector -o=jsonpath='{.data.config}' > inject-config.yaml
      $ istioctl kube-inject -f deployment.yaml -o deployment-injected.yaml --injectConfigFile inject-config.yaml
      $ kubectl apply -f deployment-injected.yaml
      

Operational details

The Istio CNI plugin handles Kubernetes pod create and delete events and does the following:

1. Identify Istio user application pods with Istio sidecars requiring traffic redirection
2. Perform pod network namespace configuration to redirect traffic to/from the Istio sidecar

Identifying pods requiring traffic redirection

The Istio CNI plugin identifies pods requiring traffic redirection to/from the accompanying Istio proxy sidecar by checking that the pod meets all of the following conditions:

1. The pod is NOT in a Kubernetes namespace in the configured exclude_namespaces list.
2. The pod has a container named istio-proxy.
3. The pod has more than 1 container.
4. The pod has no annotation with key sidecar.istio.io/inject OR the value of the annotation is true.

Traffic redirection parameters

To redirect traffic in the application pod’s network namespace to/from the Istio proxy sidecar, the Istio CNI plugin configures the namespace’s iptables. The following table describes the parameters to the redirect functionality. To override the default values for the parameters, set the corresponding application pod annotation key.

Logging

The Istio CNI plugin runs in the container runtime process space. Due to this, the kubelet process writes the plugin’s log entries into its log.

Compatibility with application init containers

The Istio CNI plugin may cause networking connectivity problems for any application initContainers. When using Istio CNI, kubelet starts an injected pod with the following steps:

1. The Istio CNI plugin sets up traffic redirection to the Istio sidecar proxy within the pod.
2. All init containers execute and complete successfully.
3. The Istio sidecar proxy starts in the pod along with the pod’s other containers.

Init containers execute before the sidecar proxy starts, which can result in traffic loss during their execution. Avoid this traffic loss with one or both of the following settings:

• Set the traffic.sidecar.istio.io/excludeOutboundIPRanges annotation to disable redirecting traffic to any CIDRs the init containers communicate with.
• Set the traffic.sidecar.istio.io/excludeOutboundPorts annotation to disable redirecting traffic to the specific outbound ports the init containers use.

Compatibility with other CNI plugins

The Istio CNI plugin maintains compatibility with the same set of CNI plugins as the current istio-init container which requires the NET_ADMIN and NET_RAW capabilities.

The Istio CNI plugin operates as a chained CNI plugin. This means its configuration is added to the existing CNI plugins configuration as a new configuration list element. See the CNI specification reference for further details. When a pod is created or deleted, the container runtime invokes each plugin in the list in order. The Istio CNI plugin only performs actions to setup the application pod’s traffic redirection to the injected Istio proxy sidecar (using iptables in the pod’s network namespace).