Security

Q: How can I enable/disable mTLS encryption after I installed Istio?

Starting with Istio 0.8, authentication policy can be used to change mutual TLS setting at run time, without needing to reinstall Istio.

Before 0.8, the most straightforward way to enable/disable mutual TLS is by entirely uninstalling and re-installing Istio.

If you are an advanced user and understand the risks you can also do the following:

$ kubectl edit configmap -n istio-system istio

comment out or uncomment authPolicy: MUTUAL_TLS to toggle mutual TLS and then

$ kubectl delete pods -n istio-system -l istio=pilot

to restart Pilot, after a few seconds (depending on your *RefreshDelay) your Envoy proxies will have picked up the change from Pilot. During that time your services may be unavailable.

Q: Can Istio mutual TLS enabled services communicate with services without Istio?

Starting with Istio 0.8, a service with Istio mutual TLS enabled can talk to a service without Istio. Mutual TLS is enabled via authentication policy and this only specifies the service behavior as a server, not client, which means a mutual TLS enabled service will still send http traffic (not mutual TLS) to others unless you explicitly specify it with destination rule.

However, unless a service without Istio can present a valid certificate, which is less likely to happen, a service without Istio cannot talk to a service with Istio mutual TLS enabled and this is the expected behavior of ‘mutual TLS’.

Q: Can I enable Istio Auth with some services while disable others in the same cluster?

Starting with Istio 0.8, you can use authentication policy to enable (or disable) mutual TLS per service. For example, the policy below will disable mutual TLS on port 9080 for service details

cat <<EOF | istioctl create -f -
apiVersion: "authentication.istio.io/v1alpha1"
kind: "Policy"
metadata:
  name: "example"
spec:
  targets:
  - name: details
    ports:
    - number: 9080
  peers:
EOF

For older versions of Istio (but newer than 0.3), you can use service-level annotations to disable (or enable) Istio Auth for a particular service and port pair. The annotation key should be auth.istio.io/{port_number}, and the value should be NONE (to disable), or MUTUAL_TLS (to enable).

Example: disable Istio Auth on port 9080 for service details.

kind: Service
metadata:
name: details
labels:
  app: details
annotations:
  auth.istio.io/9080: NONE

Q: How can I use Kubernetes liveness and readiness for service health check with Istio Auth enabled?

If Istio Auth is enabled, http and tcp health check from kubelet will not work since they do not have Istio Auth issued certs. A workaround is to use a liveness command for health check, e.g., one can install curl in the service pod and curl itself within the pod. The Istio team is actively working on a solution.

An example of readinessProbe:

livenessProbe:
exec:
  command:
  - curl
  - -f
  - http://localhost:8080/healthz # Replace port and URI by your actual health check
initialDelaySeconds: 10
periodSeconds: 5

Q: Can I access the Kubernetes API Server with Auth enabled?

The Kubernetes API server does not support mutual TLS authentication, so strictly speaking: no. However, if you use version 0.3 or later, see next question to learn how to disable mTLS in upstream config on clients side so they can access API server.

Q: How to disable mutual TLS on clients to access the Kubernetes API Server (or any control services that don't have Istio sidecar)?

This issue occurs when Istio is installed with global mutual TLS enabled. Using authentication policy, mutual TLS can be enabled selectively per service, hence can avoid this problem. Discussion below are for releases before 0.8.

Starting with release 0.3, edit the mtlsExcludedServices list in Istio config map to contain the fully-qualified name of the API server (and any other control services for that matter). The default value of mtlsExcludedServices already contains kubernetes.default.svc.cluster.local, which is the default service name of the Kubernetes API server.

For a quick reference, here are commands to edit Istio configmap and to restart pilot.

$ kubectl edit configmap -n istio-system istio
$ kubectl delete pods -n istio-system -l istio=pilot

Do not use this approach to disable mutual TLS for services that are managed by Istio (i.e. using Istio sidecar). Instead, use service-level annotations to overwrite the authentication policy (see above).

Q: How to configure the lifetime for Istio certificates?

For the workloads running in Kubernetes, the lifetime of their Istio certificates is controlled by the workload-cert-ttl flag on Citadel. The default value is 19 hours. This value should be no greater than max-workload-cert-ttl of Citadel.

Citadel uses a flag max-workload-cert-ttl to control the maximum lifetime for Istio certificates issued to workloads. The default value is 7 days. If workload-cert-ttl on Citadel or node agent is greater than max-workload-cert-ttl, Citadel will fail issuing the certificate.

Modify the istio-auth.yaml file to customize the Citadel configuration. The following modification specifies that the Istio certificates for workloads running in Kubernetes has 1 hours lifetime. Besides that, the maximum allowed Istio certificate lifetime is 48 hours.

...
kind: Deployment
...
metadata:
  name: istio-citadel
  namespace: istio-system
spec:
  ...
  template:
    ...
    spec:
      ...
      containers:
      - name: citadel
        ...
        args:
          - --workload-cert-ttl=1h # Lifetime of certificates issued to workloads in Kubernetes.
          - --max-workload-cert-ttl=48h # Maximum lifetime of certificates issued to workloads by Citadel.

For the workloads running on VMs and bare metal hosts, the lifetime of their Istio certificates is specified by the workload-cert-ttl flag on each node agent. The default value is also 19 hours. This value should be no greater than max-workload-cert-ttl of Citadel.

To customize this configuration, the argument for the node agent service should be modified. After setting up the machines for Istio mesh expansion, modify the file /lib/systemd/system/istio-auth-node-agent.service on the VMs or bare metal hosts:

...
[Service]
ExecStart=/usr/local/bin/node_agent --workload-cert-ttl=24h # Specify certificate lifetime for workloads on this machine.
Restart=always
StartLimitInterval=0
RestartSec=10
...

The above configuration specifies that the Istio certificates for workloads running on this VM or bare metal host will have 24 hours lifetime.

After configuring the service, restart the node agent by running systemctl daemon-reload.

Q: Does Istio Auth support authorization?

Yes. Starting from Istio 0.5 release, we provide Role Based Access Control for services in Istio mesh. Learn more.

Q: Does Istio Auth use Kubernetes secrets?

Yes. The key and certificate distribution in Istio Auth is based on Kubernetes secrets.

Secrets have known security risks. The Kubernetes team is working on several features to improve Kubernetes secret security, from secret encryption to node-level access control. And as of version 1.6, Kubernetes introduces RBAC authorization, which can provide fine-grained secrets management.

Q: Is the secret encrypted for workload key and cert?

By default, they are base64 encoded but not encrypted. However, the secret encryption feature is supported in Kubernetes and you can do it by following the instruction.

Notice that this feature is not enabled yet in Google Container Engine (GKE). While the data may not be encrypted inside the etcd running on the master node, the contents of the master node itself are encrypted, see here for more info.

Q: How to configure Istio Ingress to only accept TLS traffic?

By following the instructions on Adding a secure port to a gateway and on Disabling an HTTP port in a gateway Istio Ingress can be secured to only accept TLS traffic.

Q: Can I install Istio sidecar for HTTPS services?

Yes, you can. It works both with mutual TLS enabled and disabled. Refer to how Istio mTLS works with HTTPS services for more information.