Enabling Rate Limits using Envoy
This task shows you how to use Envoy’s native rate limiting to dynamically limit the traffic to an Istio
service. In this task, you will apply a global rate-limit for the productpage
service through ingress gateway that allows
1 requests per minute across all instances of the service. Additionally, you will apply a local rate-limit for each
individual productpage
instance that will allow 4 requests per minute. In this way, you will ensure that the productpage
service handles a maximum of 1 request per minute through the ingress gateway, but each productpage
instance can handle
up to 4 requests per minute, allowing for any in-mesh traffic.
Before you begin
Setup Istio in a Kubernetes cluster by following the instructions in the Installation Guide.
Deploy the Bookinfo sample application.
Rate limits
Envoy supports two kinds of rate limiting: global and local. Global rate limiting uses a global gRPC rate limiting service to provide rate limiting for the entire mesh. Local rate limiting is used to limit the rate of requests per service instance. Local rate limiting can be used in conjunction with global rate limiting to reduce load on the global rate limiting service.
In this task you will configure Envoy to rate limit traffic to a specific path of a service using both global and local rate limits.
Global rate limit
Envoy can be used to set up global rate limits for your mesh. Global rate limiting in Envoy uses a gRPC API for requesting quota from a rate limiting service. A reference implementation of the API, written in Go with a Redis backend, is used below.
Use the following configmap to configure the reference implementation to rate limit requests to the path
/productpage
at 1 req/min, a valueapi
for the coming advanced example, and all other requests at 100 req/min.$ kubectl apply -f - <<EOF apiVersion: v1 kind: ConfigMap metadata: name: ratelimit-config data: config.yaml: | domain: ratelimit descriptors: - key: PATH value: "/productpage" rate_limit: unit: minute requests_per_unit: 1 - key: PATH value: "api" rate_limit: unit: minute requests_per_unit: 2 - key: PATH rate_limit: unit: minute requests_per_unit: 100 EOF
Create a global rate limit service which implements Envoy’s rate limit service protocol. As a reference, a demo configuration can be found here, which is based on a reference implementation provided by Envoy.
$ kubectl apply -f @samples/ratelimit/rate-limit-service.yaml@
Apply an
EnvoyFilter
to theingressgateway
to enable global rate limiting using Envoy’s global rate limit filter.The patch inserts the
envoy.filters.http.ratelimit
global envoy filter into theHTTP_FILTER
chain. Therate_limit_service
field specifies the external rate limit service,outbound|8081||ratelimit.default.svc.cluster.local
in this case.$ kubectl apply -f - <<EOF apiVersion: networking.istio.io/v1alpha3 kind: EnvoyFilter metadata: name: filter-ratelimit namespace: istio-system spec: workloadSelector: # select by label in the same namespace labels: istio: ingressgateway configPatches: # The Envoy config you want to modify - applyTo: HTTP_FILTER match: context: GATEWAY listener: filterChain: filter: name: "envoy.filters.network.http_connection_manager" subFilter: name: "envoy.filters.http.router" patch: operation: INSERT_BEFORE # Adds the Envoy Rate Limit Filter in HTTP filter chain. value: name: envoy.filters.http.ratelimit typed_config: "@type": type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit # domain can be anything! Match it to the ratelimter service config domain: ratelimit failure_mode_deny: true timeout: 10s rate_limit_service: grpc_service: envoy_grpc: cluster_name: outbound|8081||ratelimit.default.svc.cluster.local authority: ratelimit.default.svc.cluster.local transport_api_version: V3 EOF
Apply another
EnvoyFilter
to theingressgateway
that defines the route configuration on which to rate limit. This adds rate limit actions for any route from a virtual host namedbookinfo.com:80
.$ kubectl apply -f - <<EOF apiVersion: networking.istio.io/v1alpha3 kind: EnvoyFilter metadata: name: filter-ratelimit-svc namespace: istio-system spec: workloadSelector: labels: istio: ingressgateway configPatches: - applyTo: VIRTUAL_HOST match: context: GATEWAY routeConfiguration: vhost: name: "" route: action: ANY patch: operation: MERGE # Applies the rate limit rules. value: rate_limits: - actions: # any actions in here - request_headers: header_name: ":path" descriptor_key: "PATH" EOF
Global rate limit advanced case
This example uses regex to match /api/*
uri
and defines a rate limit action inserted at the route level
using the VirtualService http name. The PATH value api
inserted in the prior example comes into play.
Change VirtualService so the prefix
/api/v1/products
is moved to a route calledapi
:$ kubectl apply -f - <<EOF apiVersion: networking.istio.io/v1beta1 kind: VirtualService metadata: name: bookinfo spec: gateways: - bookinfo-gateway hosts: - '*' http: - match: - uri: exact: /productpage - uri: prefix: /static - uri: exact: /login - uri: exact: /logout route: - destination: host: productpage port: number: 9080 - match: - uri: prefix: /api/v1/products route: - destination: host: productpage port: number: 9080 name: api EOF
Apply an EnvoyFilter to add the rate limits action at the route level on any 1 to 99 product:
$ kubectl apply -f - <<EOF apiVersion: networking.istio.io/v1alpha3 kind: EnvoyFilter metadata: name: filter-ratelimit-svc-api namespace: istio-system spec: workloadSelector: labels: istio: ingressgateway configPatches: - applyTo: HTTP_ROUTE match: context: GATEWAY routeConfiguration: vhost: name: "*:8080" route: name: "api" patch: operation: MERGE value: route: rate_limits: - actions: - header_value_match: descriptor_key: "PATH" descriptor_value: "api" headers: - name: ":path" safe_regex_match: google_re2: {} regex: "/api/v1/products/[1-9]{1,2}" EOF
Local rate limit
Envoy supports local rate limiting of L4 connections and HTTP requests. This allows you to apply rate limits at the instance level, in the proxy itself, without calling any other service.
The following EnvoyFilter
enables local rate limiting for any traffic through the productpage
service.
The HTTP_FILTER
patch inserts the envoy.filters.http.local_ratelimit
local envoy filter
into the HTTP connection manager filter chain. The local rate limit filter’s
token bucket
is configured to allow 4 requests/min. The filter is also configured to add an x-local-rate-limit
response header to requests that are blocked.
$ kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: filter-local-ratelimit-svc
namespace: istio-system
spec:
workloadSelector:
labels:
app: productpage
configPatches:
- applyTo: HTTP_FILTER
match:
context: SIDECAR_INBOUND
listener:
filterChain:
filter:
name: "envoy.filters.network.http_connection_manager"
patch:
operation: INSERT_BEFORE
value:
name: envoy.filters.http.local_ratelimit
typed_config:
"@type": type.googleapis.com/udpa.type.v1.TypedStruct
type_url: type.googleapis.com/envoy.extensions.filters.http.local_ratelimit.v3.LocalRateLimit
value:
stat_prefix: http_local_rate_limiter
token_bucket:
max_tokens: 4
tokens_per_fill: 4
fill_interval: 60s
filter_enabled:
runtime_key: local_rate_limit_enabled
default_value:
numerator: 100
denominator: HUNDRED
filter_enforced:
runtime_key: local_rate_limit_enforced
default_value:
numerator: 100
denominator: HUNDRED
response_headers_to_add:
- append: false
header:
key: x-local-rate-limit
value: 'true'
EOF
The above configuration applies local rate limiting to all vhosts/routes. Alternatively, you can restrict it to a specific route.
The following EnvoyFilter
enables local rate limiting for any traffic to port 9080 of the productpage
service.
Unlike the previous configuration, there is no token_bucket
included in the HTTP_FILTER
patch.
The token_bucket
is instead defined in the second (HTTP_ROUTE
) patch which includes a typed_per_filter_config
for the
envoy.filters.http.local_ratelimit
local envoy filter, for routes to virtual host inbound|http|9080
.
$ kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: filter-local-ratelimit-svc
namespace: istio-system
spec:
workloadSelector:
labels:
app: productpage
configPatches:
- applyTo: HTTP_FILTER
match:
context: SIDECAR_INBOUND
listener:
filterChain:
filter:
name: "envoy.filters.network.http_connection_manager"
patch:
operation: INSERT_BEFORE
value:
name: envoy.filters.http.local_ratelimit
typed_config:
"@type": type.googleapis.com/udpa.type.v1.TypedStruct
type_url: type.googleapis.com/envoy.extensions.filters.http.local_ratelimit.v3.LocalRateLimit
value:
stat_prefix: http_local_rate_limiter
- applyTo: HTTP_ROUTE
match:
context: SIDECAR_INBOUND
routeConfiguration:
vhost:
name: "inbound|http|9080"
route:
action: ANY
patch:
operation: MERGE
value:
typed_per_filter_config:
envoy.filters.http.local_ratelimit:
"@type": type.googleapis.com/udpa.type.v1.TypedStruct
type_url: type.googleapis.com/envoy.extensions.filters.http.local_ratelimit.v3.LocalRateLimit
value:
stat_prefix: http_local_rate_limiter
token_bucket:
max_tokens: 4
tokens_per_fill: 4
fill_interval: 60s
filter_enabled:
runtime_key: local_rate_limit_enabled
default_value:
numerator: 100
denominator: HUNDRED
filter_enforced:
runtime_key: local_rate_limit_enforced
default_value:
numerator: 100
denominator: HUNDRED
response_headers_to_add:
- append: false
header:
key: x-local-rate-limit
value: 'true'
EOF
Verify the results
Verify global rate limit
Send traffic to the Bookinfo sample. Visit http://$GATEWAY_URL/productpage
in your web browser or issue the following command:
$ for i in {1..2}; do curl -s "http://$GATEWAY_URL/productpage" -o /dev/null -w "%{http_code}\n"; sleep 3; done
200
429
$ for i in {1..3}; do curl -s "http://$GATEWAY_URL/api/v1/products/${i}" -o /dev/null -w "%{http_code}\n"; sleep 3; done
200
200
429
For /productpage
, you will see the first request go through but every following request within
a minute will get a 429 response. And for /api/v1/products/*
you will need to hit twice,
with any number in between 1-99, until you get the 429 response within a minute.
Verify local rate limit
Although the global rate limit at the ingress gateway limits requests to the productpage
service at 1 req/min,
the local rate limit for productpage
instances allows 4 req/min.
To confirm this, send internal productpage
requests, from the ratings
pod, using the following curl
command:
$ kubectl exec "$(kubectl get pod -l app=ratings -o jsonpath='{.items[0].metadata.name}')" -c ratings -- bash -c 'for i in {1..5}; do curl -s productpage:9080/productpage -o /dev/null -w "%{http_code}\n"; sleep 1; done'
200
200
200
200
429
You should see no more than 4 req/min go through per productpage
instance.
Cleanup
$ kubectl delete envoyfilter filter-ratelimit -nistio-system
$ kubectl delete envoyfilter filter-ratelimit-svc -nistio-system
$ kubectl delete envoyfilter filter-ratelimit-svc-api -nistio-system
$ kubectl delete envoyfilter filter-local-ratelimit-svc -nistio-system
$ kubectl delete cm ratelimit-config
$ kubectl delete -f @samples/ratelimit/rate-limit-service.yaml@