Enabling Rate Limits

This task shows you how to use Istio to dynamically limit the traffic to a service.

Before you begin

  1. Setup Istio in a Kubernetes cluster by following the instructions in the Installation Guide.

  2. Deploy the Bookinfo sample application.

    The Bookinfo sample deploys 3 versions of the reviews service:

    • Version v1 doesn’t call the ratings service.
    • Version v2 calls the ratings service, and displays each rating as 1 to 5 black stars.
    • Version v3 calls the ratings service, and displays each rating as 1 to 5 red stars.

    You need to set a default route to one of the versions. Otherwise, when you send requests to the reviews service, Istio routes requests to all available versions randomly, and sometimes the output contains star ratings and sometimes it doesn’t.

  3. Set the default version for all services to v1.

    $ kubectl apply -f @samples/bookinfo/networking/virtual-service-all-v1.yaml@

Rate limits

In this task, you configure Istio to rate limit traffic to productpage based on the IP address of the originating client. You will use X-Forwarded-For request header as the client IP address. You will also use a conditional rate limit that exempts logged in users.

For convenience, you configure the memory quota (memquota) adapter to enable rate limiting. On a production system, however, you need Redis, and you configure the Redis quota (redisquota) adapter. Both the memquota and redisquota adapters support the quota template, so the configuration to enable rate limiting on both adapters is the same.

  1. Rate limit configuration is split into 2 parts.

    • Client Side
      • QuotaSpec defines quota name and amount that the client should request.
      • QuotaSpecBinding conditionally associates QuotaSpec with one or more services.
    • Mixer Side
      • quota instance defines how quota is dimensioned by Mixer.
      • memquota adapter defines memquota adapter configuration.
      • quota rule defines when quota instance is dispatched to the memquota adapter.

    Run the following command to enable rate limits using memquota:

    $ kubectl apply -f @samples/bookinfo/policy/mixer-rule-productpage-ratelimit.yaml@

    Or

    Save the following yaml file as redisquota.yaml. Replace rate_limit_algorithm, redis_server_url with values for your configuration.

    apiVersion: "config.istio.io/v1alpha2"
    kind: redisquota
    metadata:
      name: handler
      namespace: istio-system
    spec:
      quotas:
      - name: requestcount.quota.istio-system
        maxAmount: 500
        validDuration: 1s
        bucketDuration: 500ms
        rateLimitAlgorithm: <rate_limit_algorithm>
        # The first matching override is applied.
        # A requestcount instance is checked against override dimensions.
        overrides:
        # The following override applies to 'reviews' regardless
        # of the source.
        - dimensions:
            destination: reviews
          maxAmount: 1
          validDuration: 5s
        # The following override applies to 'productpage' when
        # the source is a specific ip address.
        - dimensions:
            destination: productpage
            source: "10.28.11.20"
          maxAmount: 500
          validDuration: 1s
        # The following override applies to 'productpage' regardless
        # of the source.
        - dimensions:
            destination: productpage
          maxAmount: 2
          validDuration: 5s
        redisServerUrl: <redis_server_url>
        connectionPoolSize: 10
    ---
    apiVersion: "config.istio.io/v1alpha2"
    kind: quota
    metadata:
      name: requestcount
      namespace: istio-system
    spec:
      dimensions:
        source: request.headers["x-forwarded-for"] | "unknown"
        destination: destination.labels["app"] | destination.workload.name | "unknown"
        destinationVersion: destination.labels["version"] | "unknown"
    ---
    apiVersion: config.istio.io/v1alpha2
    kind: rule
    metadata:
      name: quota
      namespace: istio-system
    spec:
      # quota only applies if you are not logged in.
      # match: match(request.headers["cookie"], "user=*") == false
      actions:
      - handler: handler.redisquota
        instances:
        - requestcount.quota
    ---
    apiVersion: config.istio.io/v1alpha2
    kind: QuotaSpec
    metadata:
      name: request-count
      namespace: istio-system
    spec:
      rules:
      - quotas:
        - charge: 1
          quota: requestcount
    ---
    apiVersion: config.istio.io/v1alpha2
    kind: QuotaSpecBinding
    metadata:
      name: request-count
      namespace: istio-system
    spec:
      quotaSpecs:
      - name: request-count
        namespace: istio-system
      services:
      - name: productpage
        namespace: default
        #  - service: '*'  # Uncomment this to bind *all* services to request-count
    ---

    Run the following command to enable rate limits using redisquota:

    $ kubectl apply -f redisquota.yaml
  2. Confirm the memquota handler was created:

    $ kubectl -n istio-system get memquota handler -o yaml
    apiVersion: config.istio.io/v1alpha2
    kind: memquota
    metadata:
      name: handler
      namespace: istio-system
    spec:
      quotas:
      - name: requestcount.quota.istio-system
        maxAmount: 500
        validDuration: 1s
        - dimensions:
            destination: reviews
          maxAmount: 1
          validDuration: 5s
        - dimensions:
            destination: productpage
          maxAmount: 2
          validDuration: 5s

    The memquota handler defines 3 different rate limit schemes. The default, if no overrides match, is 500 requests per one second (1s). Two overrides are also defined:

    • The first is 1 request (the maxAmount field) every 5s (the validDuration field), if the destination is reviews.
    • The second is 2 requests every 5s, if the destination is productpage.

    When a request is processed, the first matching override is picked (reading from top to bottom).

    Or

    Confirm the redisquota handler was created:

    $ kubectl -n istio-system get redisquota handler -o yaml
    apiVersion: config.istio.io/v1alpha2
    kind: redisquota
    metadata:
      name: handler
      namespace: istio-system
    spec:
      connectionPoolSize: 10
      quotas:
      - name: requestcount.quota.istio-system
        maxAmount: 500
        validDuration: 1s
        bucketDuration: 500ms
        rateLimitAlgorithm: ROLLING_WINDOW
        overrides:
        - dimensions:
            destination: reviews
          maxAmount: 1
        - dimensions:
            destination: productpage
            source: 10.28.11.20
          maxAmount: 500
        - dimensions:
            destination: productpage
          maxAmount: 2

    The redisquota handler defines 4 different rate limit schemes. The default, if no overrides match, is 500 requests per one second (1s). It is using ROLLING_WINDOW algorithm for quota check and thus define bucketDuration of 500ms for ROLLING_WINDOW algorithm. Three overrides are also defined:

    • The first is 1 request (the maxAmount field), if the destination is reviews.
    • The second is 500, if the destination is productpage and source is 10.28.11.20
    • The third is 2, if the destination is productpage.

    When a request is processed, the first matching override is picked (reading from top to bottom).

  3. Confirm the quota instance was created:

    $ kubectl -n istio-system get quotas requestcount -o yaml
    apiVersion: config.istio.io/v1alpha2
    kind: quota
    metadata:
      name: requestcount
      namespace: istio-system
    spec:
      dimensions:
        source: request.headers["x-forwarded-for"] | "unknown"
        destination: destination.labels["app"] | destination.service.host | "unknown"
        destinationVersion: destination.labels["version"] | "unknown"

    The quota template defines three dimensions that are used by memquota or redisquota to set overrides on requests that match certain attributes. The destination will be set to the first non-empty value in destination.labels["app"], destination.service.host, or "unknown". For more information on expressions, see Expression Language.

  4. Confirm the quota rule was created:

    $ kubectl -n istio-system get rules quota -o yaml
    apiVersion: config.istio.io/v1alpha2
    kind: rule
    metadata:
      name: quota
      namespace: istio-system
    spec:
      actions:
      - handler: handler.memquota
        instances:
        - requestcount.quota

    The rule tells Mixer to invoke the handler.memquota\handler.redisquota handler (created above) and pass it the object constructed using the instance requestcount.quota (also created above). This maps the dimensions from the quota template to memquota or redisquota handler.

  5. Confirm the QuotaSpec was created:

    $ kubectl -n istio-system get QuotaSpec request-count -o yaml
    apiVersion: config.istio.io/v1alpha2
    kind: QuotaSpec
    metadata:
      name: request-count
      namespace: istio-system
    spec:
      rules:
      - quotas:
        - charge: "1"
          quota: requestcount

    This QuotaSpec defines the requestcount quota you created above with a charge of 1.

  6. Confirm the QuotaSpecBinding was created:

    $ kubectl -n istio-system get QuotaSpecBinding request-count -o yaml
    kind: QuotaSpecBinding
    metadata:
      name: request-count
      namespace: istio-system
    spec:
      quotaSpecs:
      - name: request-count
        namespace: istio-system
      services:
      - name: productpage
        namespace: default
      # - service: '*'

    This QuotaSpecBinding binds the QuotaSpec you created above to the services you want to apply it to. productpage is explicitly bound to request-count, note that you must define the namespace since it differs from the namespace of the QuotaSpecBinding. If the last line is uncommented, service: '*' binds all services to the QuotaSpec making the first entry redundant.

  7. Refresh the product page in your browser.

    request-count quota applies to productpage and it permits 2 requests every 5 seconds. If you keep refreshing the page you should see RESOURCE_EXHAUSTED:Quota is exhausted for: requestcount.

Conditional rate limits

In the above example we have effectively rate limited productpage at 2 rps per client IP. Consider a scenario where you would like to exempt clients from this rate limit if a user is logged in. In the bookinfo example, we use cookie user=<username> to denote a logged in user. In a realistic scenario you may use a jwt token for this purpose.

You can update the quota rule by adding a match condition based on the cookie.

$ kubectl -n istio-system edit rules quota

apiVersion: config.istio.io/v1alpha2
kind: rule
metadata:
  name: quota
  namespace: istio-system
spec:
  match: match(request.headers["cookie"], "user=*") == false
  actions:
  - handler: handler.memquota
    instances:
    - requestcount.quota

memquota or redisquota adapter is now dispatched only if user=<username> cookie is absent from the request. This ensures that a logged in user is not subject to this quota.

  1. Verify that rate limit does not apply to a logged in user.

    Log in as jason and repeatedly refresh the productpage. Now you should be able to do this without a problem.

  2. Verify that rate limit does apply when not logged in.

    Logout as jason and repeatedly refresh the productpage. You should again see RESOURCE_EXHAUSTED:Quota is exhausted for: requestcount.

Understanding rate limits

In the preceding examples you saw how Mixer applies rate limits to requests that match certain conditions.

Every named quota instance like requestcount represents a set of counters. The set is defined by a Cartesian product of all quota dimensions. If the number of requests in the last expiration duration exceed maxAmount, Mixer returns a RESOURCE_EXHAUSTED message to the Envoy proxy, and Envoy returns status HTTP 429 to the caller.

The memquota adapter uses a sliding window of sub-second resolution to enforce rate limits.

The redisquota adapter can be configured to use either the ROLLING_WINDOW or FIXED_WINDOW algorithms to enforce rate limits.

The maxAmount in the adapter configuration sets the default limit for all counters associated with a quota instance. This default limit applies if a quota override does not match the request. The memquota/redisquota adapter selects the first override that matches a request. An override need not specify all quota dimensions. In the example, the 0.2 qps override is selected by matching only three out of four quota dimensions.

If you want the policies enforced for a given namespace instead of the entire Istio mesh, you can replace all occurrences of istio-system with the given namespace.

Cleanup

  1. If using memquota, remove the memquota rate limit configuration:

    $ kubectl delete -f @samples/bookinfo/policy/mixer-rule-ratings-ratelimit.yaml@

    Or

    If using redisquota, remove the redisquota rate limit configuration:

    $ kubectl delete -f redisquota.yaml
  2. Remove the application routing rules:

    $ kubectl delete -f @samples/bookinfo/networking/virtual-service-all-v1.yaml@
  3. If you are not planning to explore any follow-on tasks, refer to the Bookinfo cleanup instructions to shutdown the application.

See also

Improving availability and reducing latency.

Provides an overview of Mixer's plug-in architecture.

Describes the policy enforcement and telemetry mechanisms.

Shows how to control access to a service using simple denials or white/black listing.