Extending Self-Signed Certificate Lifetime

3 minute read

Before version 1.3, Istio self-signed certificates had a 1 year default lifetime. If your cluster started with Istio version 1.2 or earlier, and it is using Istio self-signed certificates, you need to be mindful about the expiration date of the root certificate. The expiration of a root certificate may lead to an unexpected cluster-wide outage.

The following steps show you how to examine the remaining lifetime for your root certificate, and how to transition to a new root certificate with a 10 year lifetime.

Root transition procedure

Check when the root certificate expires:

Download this script on a machine that has kubectl access to the cluster.

$ wget https://raw.githubusercontent.com/istio/tools/release-1.8/bin/root-transition.sh
$ chmod +x root-transition.sh
$ ./root-transition.sh check-root
...
=====YOU HAVE 30 DAYS BEFORE THE ROOT CERT EXPIRES!=====

Execute the remainder of the steps prior to root certificate expiration to avoid system outages.

Execute a root certificate transition:

$ ./root-transition.sh root-transition
Create new ca cert, with trust domain as cluster.local
Wed Jun  5 19:11:15 PDT 2019 delete old ca secret
secret "istio-ca-secret" deleted
Wed Jun  5 19:11:15 PDT 2019 create new ca secret
secret/istio-ca-secret created
pod "istiod-86f88b6f6-d8hjt" deleted
Wed Jun  5 19:11:18 PDT 2019 restarted Citadel, checking status
NAME                                   READY   STATUS    RESTARTS   AGE
istiod-5d4798c786-w782z                1/1     Running   0          3s
New root certificate:
Certificate:
    Data:
        ...
        Validity
            Not Before: Jun  6 03:24:43 2019 GMT
            Not After : Jun  3 03:24:43 2029 GMT
        Subject: O = cluster.local
        ...
Your old certificate is stored as old-ca-cert.pem, and your private key is stored as ca-key.pem
Please save them safely and privately.

Verify the new workload certificates are loaded by Envoy:
Envoy proxies will retrieve the new root certificate when they rotate the workload key and certificates. Because the rotation is triggered based on the remaining lifetime of the existing certificate, with the default 24 hour workload certificate lifetime, expect the root transition to happen within the next 12 hours (within the 12 hour window, all workloads should rotate their keys and certificates). You can verify whether an Envoy has received the new certificates. The following command shows an example to check the Envoy’s certificate for a pod.
```
$ kubectl exec [YOUR_POD] -c istio-proxy -n [YOUR_NAMESPACE] -- curl http://localhost:15000/certs | head -c 1000
{
 "certificates": [
  {
   "ca_cert": [
      ...
      "valid_from": "2019-06-06T03:24:43Z",
      "expiration_time": ...
   ],
   "cert_chain": [
    {
      ...
    }
```
Inspect the valid_from value of ca_cert. If it matches the _Not_ _Before_ value in the new certificate as shown in Step 2, your Envoy has loaded the new root certificate. If you see your Envoy is not able to load the new certificate, check the health of Istiod. You may also manually restart the workloads.