Securing Gateways with HTTPS

Note: This task uses the new v1alpha3 traffic management API. The old API has been deprecated and will be removed in the next Istio release. If you need to use the old version, follow the docs here.

The Control Ingress Traffic task describes how to configure an ingress gateway to expose an HTTP endpoint of a service to external traffic. This task extends that task to enable HTTPS access to the service using either simple or mutual TLS.

Before you begin

Perform the steps in the Before you begin and Determining the ingress IP and ports sections of the Control Ingress Traffic task. After performing those steps you should have Istio and httpbin service deployed, and the environment variables INGRESS_HOST and SECURE_INGRESS_PORT set.
For macOS users, verify that you use curl compiled with the LibreSSL library:
```
$ curl --version | grep LibreSSL
curl 7.54.0 (x86_64-apple-darwin17.0) libcurl/7.54.0 LibreSSL/2.0.20 zlib/1.2.11 nghttp2/1.24.0
```
If a version of LibreSSL is printed as in the output above, your curl should work correctly with the instructions in this task. Otherwise, try some other installation of curl, for example on a Linux machine.

Generate client and server certificates and keys

For this task you can use your favorite tool to generate certificates and keys. We used a script from the https://github.com/nicholasjackson/mtls-go-example repository.

Clone the https://github.com/nicholasjackson/mtls-go-example repository:
```
$ git clone https://github.com/nicholasjackson/mtls-go-example
```
Change directory to the cloned repository:
```
$ cd mtls-go-example
```
Generate the certificates (use any password):
```
$ generate.sh httpbin.example.com <password>
```
The command will generate four directories: 1_root, 2_intermediate, 3_application and 4_client with client and server certificates you will use.

Configure a TLS ingress gateway

In this subsection you configure an ingress gateway with the port 443 to handle HTTPS traffic. You create a secret with a certificate and a private key. Then you create a Gateway definition that contains a server on the port 443.

Create a Kubernetes Secret to hold the server's certificate and private key. Use kubectl to create the secret istio-ingressgateway-certs in namespace istio-system . The Istio gateway will load the secret automatically.

The secret MUST be called istio-ingressgateway-certs in the istio-system namespace, or it will not be mounted and available to the Istio gateway.
```
$ kubectl create -n istio-system secret tls istio-ingressgateway-certs --key 3_application/private/httpbin.example.com.key.pem --cert 3_application/certs/httpbin.example.com.cert.pem
secret "istio-ingressgateway-certs" created
```
Note that by default all the service accounts in the istio-system namespace can access this secret, so the private key can be leaked. You can change the Role-Based Access Control (RBAC) rules to protect it.

Define a Gateway with a server section for the port 443.

The location of the certificate and the private key MUST be /etc/istio/ingressgateway-certs, or the gateway will fail to load them.

    cat <<EOF | istioctl create -f -
    apiVersion: networking.istio.io/v1alpha3
    kind: Gateway
    metadata:
      name: httpbin-gateway
    spec:
      selector:
        istio: ingressgateway # use istio default ingress gateway
      servers:
      - port:
          number: 443
          name: https
          protocol: HTTPS
        tls:
          mode: SIMPLE
          serverCertificate: /etc/istio/ingressgateway-certs/tls.crt
          privateKey: /etc/istio/ingressgateway-certs/tls.key
        hosts:
        - "httpbin.example.com"
    EOF

Configure routes for traffic entering via the Gateway. Define the same VirtualService as in the Control Ingress Traffic task:

    cat <<EOF | istioctl create -f -
    apiVersion: networking.istio.io/v1alpha3
    kind: VirtualService
    metadata:
      name: httpbin
    spec:
      hosts:
      - "httpbin.example.com"
      gateways:
      - httpbin-gateway
      http:
      - match:
        - uri:
            prefix: /status
        - uri:
            prefix: /delay
        route:
        - destination:
            port:
              number: 8000
            host: httpbin
    EOF

Access the httpbin service with HTTPS by sending an https request using curl to SECURE_INGRESS_PORT.

The --resolve flag is used to instruct curl to supply the SNI value “httpbin.example.com” when accessing the gateway IP over TLS. The --cacert option instructs curl to use your generated certificate to verify the server.

By sending the request to the /status/418 URL path, you get a nice visual clue that your httpbin service was indeed accessed. The httpbin service will return the 418 I'm a Teapot code.
```
$ curl -v --resolve httpbin.example.com:$SECURE_INGRESS_PORT:$INGRESS_HOST --cacert 2_intermediate/certs/ca-chain.cert.pem https://httpbin.example.com:$SECURE_INGRESS_PORT/status/418
...
Server certificate:
  subject: C=US; ST=Denial; L=Springfield; O=Dis; CN=httpbin.example.com
  start date: Jun 24 18:45:18 2018 GMT
  expire date: Jul  4 18:45:18 2019 GMT
  common name: httpbin.example.com (matched)
  issuer: C=US; ST=Denial; O=Dis; CN=httpbin.example.com
SSL certificate verify ok.
...
HTTP/2 418
...
-=[ teapot ]=-

   _...._
 .'  _ _ `.
| ."` ^ `". _,
\_;`"---"`|//
  |       ;/
  \_     _/
    `"""`
```
It may take time for the gateway definition to propagate and you may get the following error: Failed to connect to httpbin.example.com port <your secure port>: Connection refused. Wait for a minute and retry the curl call again.

Look for the Server certificate section in the output of curl, note the line about matching the common name: common name: httpbin.example.com (matched). According to the line SSL certificate verify ok in the output of curl, you can be sure that the server's certificate was verified successfully. Note the returned status of 418 and a nice drawing of a teapot.

If you need to support mutual TLS proceed to the next section.

Configure a mutual TLS ingress gateway

In this section you extend your gateway's definition from the previous section to support mutual TLS between external clients and the gateway.

For Istio 0.8.0, redeploy istio-ingressgateway with a volume to contain the CA certificate that the server will use to verify its clients.

$ kubectl apply -f <(helm template install/kubernetes/helm/istio --name istio --namespace istio-system -x charts/ingressgateway/templates/deployment.yaml --set ingressgateway.deployment.secretVolumes[0].name=ingressgateway-certs,ingressgateway.deployment.secretVolumes[0].secretName=istio-ingressgateway-certs,ingressgateway.deployment.secretVolumes[0].mountPath=/etc/istio/ingressgateway-certs,ingressgateway.deployment.secretVolumes[1].name=ingressgateway-ca-certs,ingressgateway.deployment.secretVolumes[1].secretName=istio-ingressgateway-ca-certs,ingressgateway.deployment.secretVolumes[1].mountPath=/etc/istio/ingressgateway-ca-certs)
deployment "istio-ingressgateway" configured

Create a Kubernetes Secret to hold the CA certificate, namely istio-ingressgateway-ca-certs in namespace istio-system. The Istio gateway will automatically load the secret.

The secret MUST be called istio-ingressgateway-ca-certs in the istio-system namespace, or it will not be mounted and available to the Istio gateway.
```
$ kubectl create -n istio-system secret generic istio-ingressgateway-ca-certs --from-file=2_intermediate/certs/ca-chain.cert.pem
secret "istio-ingressgateway-ca-certs" created
```

Redefine your previous Gateway while changing the tls mode to MUTUAL and specifying caCertificates:

The location of the certificate MUST be /etc/istio/ingressgateway-ca-certs, or the gateway will fail to load them. The file name of the certificate must be identical to the filename you create the secret from, in this case ca-chain.cert.pem.

    cat <<EOF | istioctl replace -f -
    apiVersion: networking.istio.io/v1alpha3
    kind: Gateway
    metadata:
      name: httpbin-gateway
    spec:
      selector:
        istio: ingressgateway # use istio default ingress gateway
      servers:
      - port:
          number: 443
          name: https
          protocol: HTTPS
        tls:
          mode: MUTUAL
          serverCertificate: /etc/istio/ingressgateway-certs/tls.crt
          privateKey: /etc/istio/ingressgateway-certs/tls.key
          caCertificates: /etc/istio/ingressgateway-ca-certs/ca-chain.cert.pem
        hosts:
        - "httpbin.example.com"
    EOF

Access the httpbin service by HTTPS as in the previous section:
```
$ curl --resolve httpbin.example.com:$SECURE_INGRESS_PORT:$INGRESS_HOST  --cacert 2_intermediate/certs/ca-chain.cert.pem https://httpbin.example.com:$SECURE_INGRESS_PORT/status/418
curl: (35) error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure
```
It may take time for the gateway definition to propagate and you may still get 418. Wait for a minute and retry the curl call again.

This time you get an error since the server refuses to accept unauthenticated requests. You have to send a client certificate and pass curl your private key for signing the request.

Resend the previous request by curl, this time passing as parameters your client certificate (the --cert option) and your private key (the --key option):

$ curl --resolve httpbin.example.com:$SECURE_INGRESS_PORT:$INGRESS_HOST  --cacert 2_intermediate/certs/ca-chain.cert.pem --cert 4_client/certs/httpbin.example.com.cert.pem --key 4_client/private/httpbin.example.com.key.pem https://httpbin.example.com:$SECURE_INGRESS_PORT/status/418

-=[ teapot ]=-

   _...._
 .'  _ _ `.
| ."` ^ `". _,
\_;`"---"`|//
  |       ;/
  \_     _/
    `"""`

This time the server performed client authentication successfully and you received the pretty teapot drawing again.

Troubleshooting

Inspect the values of the INGRESS_HOST and SECURE_INGRESS_PORT environment variables. Make sure they have valid values, according to the output of the following commands:
```
$ kubectl get svc -n istio-system
$ echo INGRESS_HOST=$INGRESS_HOST, SECURE_INGRESS_PORT=$SECURE_INGRESS_PORT
```

Verify that the key and the certificate are successfully loaded in the istio-ingressgateway pod:

$ kubectl exec -it -n istio-system $(kubectl -n istio-system get pods -l istio=ingressgateway -o jsonpath='{.items[0].metadata.name}') -- ls -al /etc/istio/ingressgateway-certs

tls.crt and tls.key should exist in the directory contents.

Check the log of istio-ingressgateway for error messages:

$ kubectl logs -n istio-system -l istio=ingressgateway

For mutual TLS, verify that the CA certificate is loaded in the istio-ingressgateway pod:

$ kubectl exec -it -n istio-system $(kubectl -n istio-system get pods -l istio=ingressgateway -o jsonpath='{.items[0].metadata.name}') -- ls -al /etc/istio/ingressgateway-ca-certs

ca-chain.cert.pem should exist in the directory contents.

For macOS users, verify that you use curl compiled with the LibreSSL library, as described in the Before you begin section.

Cleanup

Delete the Gateway configuration, the VirtualService and the secrets:

$ istioctl delete gateway httpbin-gateway
$ istioctl delete virtualservice httpbin
$ kubectl delete --ignore-not-found=true -n istio-system secret istio-ingressgateway-certs istio-ingressgateway-ca-certs

Shutdown the httpbin service:

$ kubectl delete --ignore-not-found=true -f @samples/httpbin/httpbin.yaml@