Enabling Rate Limits

This task shows you how to use Istio to dynamically limit the traffic to a service.

Before you begin

  • Setup Istio in a Kubernetes cluster by following the quick start instructions in the Installation guide.

  • Deploy the BookInfo sample application.

  • Initialize the application version routing to direct reviews service requests from test user “jason” to version v2 and requests from any other user to v3.

    istioctl create -f samples/bookinfo/kube/route-rule-reviews-test-v2.yaml
istioctl create -f samples/bookinfo/kube/route-rule-reviews-v3.yaml

    Note: if you have conflicting rule that you set in previous tasks, use istioctl replace instead of istioctl create.

Rate limits

Istio enables users to rate limit traffic to a service.

Consider ratings as an external paid service like Rotten Tomatoes® with 1qps free quota. Using Istio we can ensure that 1qps is not breached.

  1. Point your browser at the BookInfo productpage (http://$GATEWAY_URL/productpage).

    If you log in as user “jason”, you should see black ratings stars with each review, indicating that the ratings service is being called by the “v2” version of the reviews service.

    If you log in as any other user (or logout) you should see red ratings stars with each review, indicating that the ratings service is being called by the “v3” version of the reviews service.

  2. Configure a memquota adapter with rate limits.

    Save the following YAML snippet as ratelimit-handler.yaml.

    apiVersion: config.istio.io/v1alpha2
kind: memquota
metadata:
  name: handler
  namespace: istio-system
spec:
  quotas:
  - name: requestcount.quota.istio-system
    # default rate limit is 5000qps
    maxAmount: 5000
    validDuration: 1s
    # The first matching override is applied.
    # A requestcount instance is checked against override dimensions.
    overrides:
    # The following override applies to traffic from 'rewiews' version v2,
    # destined for the ratings service. The destinationVersion dimension is ignored.
    - dimensions:
        destination: ratings
        source: reviews
        sourceVersion: v2
      maxAmount: 1
      validDuration: 1s

    and then run the following command:

    istioctl create -f ratelimit-handler.yaml

    This configuration specifies a default 5000 qps rate limit. Traffic reaching the ratings service via reviews-v2 is subject to a 1qps rate limit. In our example user “jason” is routed via reviews-v2 and is therefore subject to the 1qps rate limit.

  3. Configure rate limit instance and rule

    Create a quota instance named requestcount that maps incoming attributes to quota dimensions, and create a rule that uses it with the memquota handler.

    apiVersion: config.istio.io/v1alpha2
kind: quota
metadata:
  name: requestcount
  namespace: istio-system
spec:
  dimensions:
    source: source.labels["app"] | source.service | "unknown"
    sourceVersion: source.labels["version"] | "unknown"
    destination: destination.labels["app"] | destination.service | "unknown"
    destinationVersion: destination.labels["version"] | "unknown"
---
apiVersion: config.istio.io/v1alpha2
kind: rule
metadata:
  name: quota
  namespace: istio-system
spec:
  actions:
  - handler: handler.memquota
    instances:
    - requestcount.quota

    Save the configuration as ratelimit-rule.yaml and run the following command:

    istioctl create -f ratelimit-rule.yaml

  4. Generate load on the productpage with the following command:

    while true; do curl -s -o /dev/null http://$GATEWAY_URL/productpage; done

  5. Refresh the productpage in your browser.

    If you log in as user “jason” while the load generator is running (i.e., generating more than 1 req/s), the traffic generated by your browser will be rate limited to 1qps. The reviews-v2 service is unable to access the ratings service and you stop seeing stars. For all other users the default 5000qps rate limit will apply and you will continue seeing red stars.

Conditional rate limits

In the previous example we applied a rate limit to the ratings service without regard to non-dimension attributes. It is possible to conditionally apply rate limits based on arbitrary attributes using a match condition in the quota rule.

For example, consider the following configuration:

apiVersion: config.istio.io/v1alpha2
kind: rule
metadata:
 name: quota
 namespace: istio-system
spec:
 match: source.namespace != destination.namespace
 actions:
 - handler: handler.memquota
   instances:
   - requestcount.quota

This configuration applies the quota rule to requests whose source and destination namespaces are different.

Understanding rate limits

In the preceding examples we saw how Mixer applies rate limits to requests that match certain conditions.

Every named quota instance like requestcount represents a set of counters. The set is defined by a Cartesian product of all quota dimensions. If the number of requests in the last expiration duration exceed maxAmount, Mixer returns a RESOURCE_EXHAUSTED message to the proxy. The proxy in turn returns status HTTP 429 to the caller.

The memquota adapter uses a sliding window of sub second resolution to enforce rate limits.

The maxAmount in the adapter configuration sets the default limit for all counters associated with a quota instance. This default limit applies if a quota override does not match the request. Memquota selects the first override that matches a request. An override need not specify all quota dimensions. In the ratelimit-handler.yaml example, the 1qps override is selected by matching only three out of four quota dimensions.

If you would like the above policies enforced for a given namespace instead of the entire Istio mesh, you can replace all occurrences of istio-system with the given namespace.

Cleanup

  • Remove the rate limit configuration:

    istioctl delete -f ratelimit-handler.yaml
istioctl delete -f ratelimit-rule.yaml

  • Remove the application routing rules:

    istioctl delete -f samples/bookinfo/kube/route-rule-reviews-test-v2.yaml
istioctl delete -f samples/bookinfo/kube/route-rule-reviews-v3.yaml

  • If you are not planning to explore any follow-on tasks, refer to the BookInfo cleanup instructions to shutdown the application.

What’s next