Security FAQ

You can change mutual TLS settings for your services at any time using authentication policy and destination rule. See task for more details.

The istioctl command provides an option for this purpose. You can do:

$ istioctl authn tls-check $CLIENT_POD httpbin.default.svc.cluster.local
HOST:PORT                                  STATUS     SERVER     CLIENT           AUTHN POLICY     DESTINATION RULE
httpbin.default.svc.cluster.local:8000     OK         STRICT     ISTIO_MUTUAL     /default         istio-system/default

Where $CLIENT_POD is the ID of one of the client service’s pods.

Refer to Verify mutual TLS configuration for more information.

Authentication policy can be mesh-wide (which affects all services in the mesh), namespace-wide (all services in the same namespace) or service specific. You can have policy or policies to setup mutual TLS for services in a cluster in any way as you want.

Non-Istio services cannot communicate to Istio services unless they can present a valid certificate, which is less likely to happen. This is the expected behavior for mutual TLS. However, you can override the global flag for specific namespaces or services. See task for more details.

When mutual TLS is globally enabled, the global destination rule matches all services in the cluster, whether or not these services have an Istio sidecar. This includes the Kubernetes API server, as well as any non-Istio services in the cluster. To communicate with such services from services that have an Istio sidecar, you need to set a destination rule to exempt the service. For example:

$ kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
 name: "api-server"
spec:
 host: "kubernetes.default.svc.cluster.local"
 trafficPolicy:
   tls:
     mode: DISABLE
EOF

Similarly, you can add destination rules for other non-Istio services. For more examples, see task.

If mutual TLS is enabled, HTTP and TCP health checks from the kubelet will not work without modification, since the kubelet does not have Istio-issued certificates.

As of Istio 1.1, we have several options to solve this issue.

1. Using probe rewrite to redirect liveness and readiness requests to the workload directly. Please refer to Probe Rewrite for more information.

2. Using a separate port for health checks and enabling mutual TLS only on the regular service port. Please refer to Health Checking of Istio Services for more information.

3. Using the PERMISSIVE mode for Istio services so they can accept both HTTP and mutual TLS traffic. Please keep in mind that mutual TLS is not enforced since others can communicate with the service with HTTP traffic.

4. Using a liveness command for health checks, e.g., one can install curl in the service pod and curl itself within the pod.

An example of a readiness probe:

livenessProbe:
exec:
  command:
  - curl
  - -f
  - http://localhost:8080/healthz # Replace port and URI by your actual health check
initialDelaySeconds: 10
periodSeconds: 5

For the workloads running in Kubernetes, the lifetime of their Istio certificates is controlled by the workload-cert-ttl flag on Citadel. The default value is 90 days. This value should be no greater than max-workload-cert-ttl of Citadel.

Citadel uses a flag max-workload-cert-ttl to control the maximum lifetime for Istio certificates issued to workloads. The default value is 90 days. If workload-cert-ttl on Citadel or node agent is greater than max-workload-cert-ttl, Citadel will fail issuing the certificate.

You can modify a generated manifest file to customize the Citadel configuration. The following modification specifies that the Istio certificates for workloads running in Kubernetes has 1 hours lifetime. Besides that, the maximum allowed Istio certificate lifetime is 48 hours.

...
kind: Deployment
...
metadata:
  name: istio-citadel
  namespace: istio-system
spec:
  ...
  template:
    ...
    spec:
      ...
      containers:
      - name: citadel
        ...
        args:
          - --workload-cert-ttl=1h # Lifetime of certificates issued to workloads in Kubernetes.
          - --max-workload-cert-ttl=48h # Maximum lifetime of certificates issued to workloads by Citadel.

For the workloads running on VMs and bare metal hosts, the lifetime of their Istio certificates is specified by the workload-cert-ttl flag on each node agent. The default value is also 90 days. This value should be no greater than max-workload-cert-ttl of Citadel.

To customize this configuration, the argument for the node agent service should be modified. After setting up the machines for Istio mesh expansion, modify the file /lib/systemd/system/istio-auth-node-agent.service on the VMs or bare metal hosts:

...
[Service]
ExecStart=/usr/local/bin/node_agent --workload-cert-ttl=24h # Specify certificate lifetime for workloads on this machine.
Restart=always
StartLimitInterval=0
RestartSec=10
...

The above configuration specifies that the Istio certificates for workloads running on this VM or bare metal host will have 24 hours lifetime.

After configuring the service, restart the node agent by running systemctl daemon-reload.

You may find MySQL can’t connect after installing Istio. This is because of PERMISSIVE mode, which is enabled in the demo configuration profile, does not work with MySQL. You may see error messages such as ERROR 2013 (HY000): Lost connection to MySQL server at 'reading initial communication packet', system error: 0.

There have two options to solve the problem.

1. Disable Mutual TLS.

   Choose this option if you don’t want Istio mutual TLS. You achieve this by disabling mutual TLS on the MySQL service explicitly.

   $ kubectl apply -f - <<EOF
   apiVersion: "authentication.istio.io/v1alpha1"
   kind: "Policy"
   metadata:
     name: mysql-nomtls-authn
   spec:
     targets:
     - name: YOUR-MYSQL-SERVICE     # The name of *your* K8s Service
   EOF
   

2. Enable mutual TLS in STRICT mode.

   If you want mutual TLS protection for MySQL, enable mutual TLS using a destination rule and an authentication policy.

   $ kubectl apply -f - <<EOF
   apiVersion: "authentication.istio.io/v1alpha1"
   kind: "Policy"
   metadata:
     name: mysql-mtls-authn
   spec:
     targets:
     - name: YOUR-MYSQL-SERVICE     # The name of *your* K8s Service
     peers:
     - mtls:
         mode: STRICT
   ---
   apiVersion: networking.istio.io/v1alpha3
   kind: DestinationRule
   metadata:
     name: mysql-mtls-dr
   spec:
     host: YOUR-MYSQL-SERVICE     # The name of *your* K8s Service
     trafficPolicy:
       tls:
         mode: ISTIO_MUTUAL
   EOF
   

Yes. Istio provides authorization features for both HTTP and plain TCP services in the mesh. Learn more.

Yes. The key and certificate distribution in Istio Authentication is based on Kubernetes secrets.

Secrets have known security risks. The Kubernetes team is working on several features to improve Kubernetes secret security, from secret encryption to node-level access control. And as of version 1.6, Kubernetes introduces RBAC authorization, which can provide fine-grained secrets management.

By default, they are base64 encoded but not encrypted. However, the secret encryption feature is supported in Kubernetes and you can do it by following the instruction.

Notice that this feature is not enabled yet in Google Container Engine (GKE). While the data may not be encrypted inside the etcd running on the master node, the contents of the master node itself are encrypted, see here for more info.

By following the instructions in the Secure Ingress Traffic task, Istio Ingress can be secured to only accept TLS traffic.

Yes, you can. It works both with mutual TLS enabled and disabled. Refer to how mutual TLS works with HTTPS services for more information.