Control Ingress Traffic

In a Kubernetes environment, the Kubernetes Ingress Resource allows users to specify services that should be exposed outside the cluster. For traffic entering an Istio service mesh, however, an Istio-aware ingress controller is needed to allow Istio features, for example, monitoring and route rules, to be applied to traffic entering the cluster.

Istio provides an envoy-based ingress controller that implements very limited support for standard Kubernetes Ingress resources as well as full support for an alternative specification, Istio Gateway. Using a Gateway is the recommended approach for configuring ingress traffic for Istio services. It is significantly more functional, not to mention the only option for non-Kubernetes environments.

This task describes how to configure Istio to expose a service outside of the service mesh using either specification.

Before you begin

  • Setup Istio by following the instructions in the Installation guide.

  • Make sure your current directory is the istio directory.

  • Start the httpbin sample, which will be used as the destination service to be exposed externally.

    If you installed the Istio-Initializer, do

    kubectl apply -f samples/httpbin/httpbin.yaml
    

    Without the Istio-Initializer:

    kubectl apply -f <(istioctl kube-inject -f samples/httpbin/httpbin.yaml)
    
  • Generate a certificate and key that will be used to demonstrate a TLS-secured gateway

    A private key and certificate can be created for testing using OpenSSL.

     openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /tmp/tls.key -out /tmp/tls.crt -subj "/CN=foo.bar.com"
    

Note: This is still a WIP and not working yet.

An Istio Gateway is the preferred model for configuring ingress traffic in Istio. An ingress Gateway describes a load balancer operating at the edge of the mesh receiving incoming HTTP/TCP connections. It configures exposed ports, protocols, etc., but, unlike Kubernetes Ingress Resources, does not include any traffic routing configuration. Traffic routing for ingress traffic is instead configured using Istio routing rules, exactly in the same was as for internal service requests.

Configuring a Gateway

  1. Create an Istio Gateway

    cat <<EOF | kubectl create -f -
    apiVersion: networking.istio.io/v1alpha3
    kind: Gateway
    metadata:
      name: httpbin-gateway
    spec:
      servers:
      - port:
          number: 80
          name: http
      - port:
          number: 443
          name: https
        tls:
          mode: SIMPLE
          serverCertificate: /tmp/tls.crt
          privateKey: /tmp/tls.key
    EOF
    

    Notice that a single Gateway specification can configure multiple ports, a simple HTTP (port 80) and secure HTTPS (port 443) in our case.

  2. Configure routes for traffic entering via the Gateway

    cat <<EOF | kubectl create -f -
    apiVersion: networking.istio.io/v1alpha3
    kind: VirtualService
    metadata:
      name: httpbin
    spec:
      hosts:
      - httpbin
      gateways:
      - httpbin-gateway
      http:
      - match:
          uri:
            prefix: /status
        route:
        - destination:
            port:
              number: 8000
            name: httpbin
      - match:
          uri:
            prefix: /delay
        route:
        - destination:
            port:
              number: 8000
            name: httpbin
    EOF
    

    Here we’ve created a VirtualService configuration for the httpbin service, containing two route rules that allow traffic for paths /status and /delay. The gateways list specifies that only requests through our httpbin-gateway are allowed. All other external requests will be rejected with a 404 response.

    Note that in this configuration internal requests from other services in the mesh are not subject to these rules, but instead will simply default to round-robin routing. To apply these (or other rules) to internal calls, we could add the special value mesh to the list of gateways.

Verifying a Gateway

The proxy instances implementing a particular Gateway configuration can be specified using a selector field. If not specified, as in our case, the Gateway will be implemented by the default istio-ingress controller. Therefore, to test our Gateway we will send requests to the istio-ingress service.

  1. Get the ingress controller pod’s hostIP:

    kubectl -n istio-system get po -l istio=ingress -o jsonpath='{.items[0].status.hostIP}'
    
    169.47.243.100
    
  2. Get the istio-ingress service’s nodePorts for port 80 and 443:

    kubectl -n istio-system get svc istio-ingress
    
    NAME            CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGE
    istio-ingress   10.10.10.155   <pending>     80:31486/TCP,443:32254/TCP   32m
    
    export INGRESS_HOST=169.47.243.100:31486
    export SECURE_INGRESS_HOST=169.47.243.100:32254
    
  3. Access the httpbin service with either HTTP or HTTPS using curl:

    curl -I http://$INGRESS_HOST/status/200
    
    HTTP/1.1 200 OK
    server: envoy
    date: Mon, 29 Jan 2018 04:45:49 GMT
    content-type: text/html; charset=utf-8
    access-control-allow-origin: *
    access-control-allow-credentials: true
    content-length: 0
    x-envoy-upstream-service-time: 48
    
    curl -I -k https://$SECURE_INGRESS_HOST/status/200
    
    HTTP/1.1 200 OK
    server: envoy
    date: Mon, 29 Jan 2018 04:45:49 GMT
    content-type: text/html; charset=utf-8
    access-control-allow-origin: *
    access-control-allow-credentials: true
    content-length: 0
    x-envoy-upstream-service-time: 96
    
  4. Access any other URL that has not been explicitly exposed. You should see a HTTP 404 error:

    curl -I http://$INGRESS_HOST/headers
    
    HTTP/1.1 404 Not Found
    date: Mon, 29 Jan 2018 04:45:49 GMT
    server: envoy
    content-length: 0
    
    curl -I https://$SECURE_INGRESS_HOST/headers
    
    HTTP/1.1 404 Not Found
    date: Mon, 29 Jan 2018 04:45:49 GMT
    server: envoy
    content-length: 0
    

Configuring ingress using a Kubernetes Ingress resource

An Istio Ingress specification is based on the standard Kubernetes Ingress Resource specification, with the following differences:

  1. Istio Ingress specification contains a kubernetes.io/ingress.class: istio annotation.

  2. All other annotations are ignored.

  3. Path syntax is c++11 regex format

Note that Ingress traffic is not affected by routing rules configured for a backend (i.e., an Istio VirtualService cannot be combined with an Ingress specification). Traffic splitting, fault injection, mirroring, header match, etc., will not work for ingress traffic. A DestinationRule associated with the backend service will, however, work as expected.

The servicePort field in the Ingress specification can take a port number (integer) or a name. The port name must follow the Istio port naming conventions (e.g., grpc-*, http2-*, http-*, etc.) in order to function properly. The name used must match the port name in the backend service declaration.

Configuring simple Ingress

  1. Create a basic Ingress specification for the httpbin service

    cat <<EOF | kubectl create -f -
    apiVersion: extensions/v1beta1
    kind: Ingress
    metadata:
      name: simple-ingress
      annotations:
        kubernetes.io/ingress.class: istio
    spec:
      rules:
      - http:
          paths:
          - path: /status/*
            backend:
              serviceName: httpbin
              servicePort: 8000
          - path: /delay/*
            backend:
              serviceName: httpbin
              servicePort: 8000
    EOF
    

Verifying simple Ingress

  1. Determine the ingress URL:

    • If your cluster is running in an environment that supports external load balancers, use the ingress’ external address:

      kubectl get ingress simple-ingress -o wide
      
      NAME             HOSTS     ADDRESS                 PORTS     AGE
      simple-ingress   *         130.211.10.121          80        1d
      
      export INGRESS_HOST=130.211.10.121
      
    • If load balancers are not supported, use the ingress controller pod’s hostIP:

      kubectl -n istio-system get po -l istio=ingress -o jsonpath='{.items[0].status.hostIP}'
      
      169.47.243.100
      

      along with the istio-ingress service’s nodePort for port 80:

      kubectl -n istio-system get svc istio-ingress
      
      NAME            CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGE
      istio-ingress   10.10.10.155   <pending>     80:31486/TCP,443:32254/TCP   32m
      
      export INGRESS_HOST=169.47.243.100:31486
      
  2. Access the httpbin service using curl:

    curl -I http://$INGRESS_HOST/status/200
    
    HTTP/1.1 200 OK
    server: envoy
    date: Mon, 29 Jan 2018 04:45:49 GMT
    content-type: text/html; charset=utf-8
    access-control-allow-origin: *
    access-control-allow-credentials: true
    content-length: 0
    x-envoy-upstream-service-time: 48
    
  3. Access any other URL that has not been explicitly exposed. You should see a HTTP 404 error

    curl -I http://$INGRESS_HOST/headers
    
    HTTP/1.1 404 Not Found
    date: Mon, 29 Jan 2018 04:45:49 GMT
    server: envoy
    content-length: 0
    

Configuring secure Ingress (HTTPS)

  1. Create a Kubernetes Secret to hold the key/cert

    Create the secret istio-ingress-certs in namespace istio-system using kubectl. The Istio ingress controller will automatically load the secret.

    Note: the secret MUST be called istio-ingress-certs in the istio-system namespace, or it will not be mounted and available to the Istio ingress controller.

    kubectl create -n istio-system secret tls istio-ingress-certs --key /tmp/tls.key --cert /tmp/tls.crt
    

    Note that by default all service accounts in the istio-system namespace can access this ingress key/cert, which risks leaking the key/cert. You can change the Role-Based Access Control (RBAC) rules to protect them. See (Link TBD) for details.

  2. Create the Ingress specification for the httpbin service

    cat <<EOF | kubectl create -f -
    apiVersion: extensions/v1beta1
    kind: Ingress
    metadata:
      name: secure-ingress
      annotations:
        kubernetes.io/ingress.class: istio
    spec:
      tls:
        - secretName: istio-ingress-certs # currently ignored
      rules:
      - http:
          paths:
          - path: /status/*
            backend:
              serviceName: httpbin
              servicePort: 8000
          - path: /delay/*
            backend:
              serviceName: httpbin
              servicePort: 8000
    EOF
    

    Note: Because SNI is not yet supported, Envoy currently only allows a single TLS secret in the ingress. That means the secretName field in ingress resource is not used.

Verifying secure Ingress

  1. Determine the ingress URL:

    • If your cluster is running in an environment that supports external load balancers, use the ingress’ external address:

      kubectl get ingress secure-ingress -o wide
      
      NAME             HOSTS     ADDRESS                 PORTS     AGE
      secure-ingress   *         130.211.10.121          80        1d
      
      export SECURE_INGRESS_HOST=130.211.10.121
      
    • If load balancers are not supported, use the ingress controller pod’s hostIP:

      kubectl -n istio-system get po -l istio=ingress -o jsonpath='{.items[0].status.hostIP}'
      
      169.47.243.100
      

      along with the istio-ingress service’s nodePort for port 443:

      kubectl -n istio-system get svc istio-ingress
      
      NAME            CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGE
      istio-ingress   10.10.10.155   <pending>     80:31486/TCP,443:32254/TCP   32m
      
      export SECURE_INGRESS_HOST=169.47.243.100:32254
      
  2. Access the httpbin service using curl:

    curl -I -k https://$SECURE_INGRESS_HOST/status/200
    
    HTTP/1.1 200 OK
    server: envoy
    date: Mon, 29 Jan 2018 04:45:49 GMT
    content-type: text/html; charset=utf-8
    access-control-allow-origin: *
    access-control-allow-credentials: true
    content-length: 0
    x-envoy-upstream-service-time: 96
    
  3. Access any other URL that has not been explicitly exposed. You should see a HTTP 404 error

    curl -I -k https://$SECURE_INGRESS_HOST/headers
    
    HTTP/1.1 404 Not Found
    date: Mon, 29 Jan 2018 04:45:49 GMT
    server: envoy
    content-length: 0
    

Understanding what happened

Gateway or Ingress configuration resources allow external traffic to enter the Istio service mesh and make the traffic management and policy features of Istio available for edge services.

In the preceding steps we created a service inside the Istio service mesh and showed how to expose both HTTP and HTTPS endpoints of the service to external traffic. Using an Istio Gateway provides significantly more functionality and is recommended. Using a Kubernetes Ingress, however, is also supported and may be especially useful when moving existing Kubernetes applications to Istio.

Cleanup

  1. Remove the Gateway configuration.

    kubectl delete gateway httpbin-gateway
    
  2. Remove the Ingress configuration.

    kubectl delete ingress simple-ingress secure-ingress 
    
  3. Remove the routing rule and secret.

    istioctl delete virtualservice httpbin
    kubectl delete -n istio-system secret istio-ingress-certs
    
  4. Shutdown the httpbin service.

    kubectl delete -f samples/httpbin/httpbin.yaml
    

What’s next