Envoy Global Rate Limit Service Setup

Envoy’s local rate limit filter counts inside one proxy process; the global filter asks an external service that owns a shared counter, which is the only way a mesh of thirty sidecars enforces one number. This setup guide sits under edge and gateway enforcement, and covers the three pieces that must agree — the filter’s actions, the service’s domain configuration, and the Redis behind it — plus the failure policy that decides what a mesh does when the counter service is unreachable.

The problem in concrete numbers

A payments service runs 24 pods, each with an Envoy sidecar, behind an ingress with four more Envoys. You want 50 requests per second per customer to the /v1/charges route. Configure the local filter at 50 and each of 28 proxies enforces it separately: a customer distributed across the fleet gets up to 1,400 rps, and the number you promised in the contract is off by a factor of 28. The global filter replaces those 28 independent counters with one call to a service holding a single Redis key per (customer, route) pair — at the cost of roughly 0.5–2 ms per request and a new dependency in the hot path.

Envoy global rate limiting call path A request reaches an Envoy proxy, which builds descriptors from route actions and sends them over gRPC to the rate limit service; the service looks up the matching domain rule, increments a Redis counter, and returns OK or OVER_LIMIT, which Envoy turns into a proxied request or a 429. One request, one descriptor set, one shared counter client request Envoy proxy builds descriptors rate limit service matches domain rules Redis INCR per window key gRPC OK: proxy upstream OVER_LIMIT: 429 28 proxies, one counter — the descriptor is the contract between them

How descriptors work

A descriptor is an ordered list of key–value pairs Envoy extracts from the request and sends to the service. The service matches them against a tree in its domain configuration; the first matching leaf supplies the limit. Getting this mental model right removes most of the confusion: Envoy does not know your limits, and the service does not know your routes. The descriptor is the only vocabulary they share.

Envoy action Descriptor entry produced Typical use
request_headers { key: customer, value: <header value> } Per-API-key or per-tenant limits
remote_address { key: remote_address, value: <ip> } Per-IP shielding
destination_cluster { key: destination_cluster, value: <name> } Per-upstream protection
generic_key { key: <descriptor_key>, value: <descriptor_value> } Route or endpoint class labels
header_value_match { key: header_match, value: <label> } Method or path-pattern classes
metadata { key: <name>, value: <dynamic metadata> } Values computed by an earlier filter (e.g. authenticated plan)

Order matters. [route, customer] and [customer, route] are different descriptors and match different branches of the service configuration; a mismatch produces a silent pass-through, because an unmatched descriptor is not an error — it simply has no limit.

How a descriptor list matches the service configuration tree The proxy emits an ordered descriptor of route class then customer; the service walks its configuration tree in the same order, taking a specific customer override if one exists and otherwise the generic per-customer limit. Descriptor order is the matching order emitted: route_class=charges, customer=acct_7 config level 1: key route_class, value charges level 2: customer=acct_enterprise_7 → 500/s level 2 fallback: any customer → 50/s (match) reverse the order and nothing matches, so nothing is limited — silently an unmatched descriptor is not an error, which is why misconfiguration fails open

Step-by-step setup

yaml
# envoy.yaml — HTTP filter + route actions
http_filters:
  - name: envoy.filters.http.ratelimit
    typed_config:
      "@type": type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
      domain: api_gateway            # must match the service config's domain
      failure_mode_deny: false       # explicit: allow traffic if the service is down
      timeout: 0.03s                 # 30 ms — never let the limiter own your latency
      enable_x_ratelimit_headers: DRAFT_VERSION_03
      rate_limit_service:
        grpc_service:
          envoy_grpc: { cluster_name: ratelimit_cluster }
        transport_api_version: V3

route_config:
  virtual_hosts:
    - name: api
      domains: ["*"]
      routes:
        - match: { prefix: "/v1/charges" }
          route:
            cluster: charges_service
            rate_limits:
              # Descriptor: [ route_class=charges, customer=<X-API-Key> ]
              - actions:
                  - generic_key: { descriptor_key: route_class, descriptor_value: charges }
                  - request_headers:
                      header_name: x-api-key
                      descriptor_key: customer
                      skip_if_absent: false     # no key => no descriptor => no limit
yaml
# ratelimit service config: config/api_gateway.yaml
domain: api_gateway
descriptors:
  - key: route_class
    value: charges
    descriptors:
      # Specific customers first — deepest match wins.
      - key: customer
        value: acct_enterprise_7
        rate_limit: { unit: second, requests_per_unit: 500 }
      # Then the generic per-customer bucket: one counter per distinct value.
      - key: customer
        rate_limit: { unit: second, requests_per_unit: 50 }
  # Catch-all for every other route class.
  - key: route_class
    rate_limit: { unit: second, requests_per_unit: 200 }
yaml
# The service itself, with Redis as the shared counter.
env:
  - { name: USE_STATSD, value: "true" }
  - { name: REDIS_SOCKET_TYPE, value: tcp }
  - { name: REDIS_URL, value: "redis-ratelimit:6379" }
  - { name: REDIS_POOL_SIZE, value: "20" }
  - { name: LOCAL_CACHE_SIZE_IN_BYTES, value: "1048576" }  # caches OVER_LIMIT keys
  - { name: NEAR_LIMIT_RATIO, value: "0.8" }
  - { name: RUNTIME_ROOT, value: /data }
  - { name: LOG_LEVEL, value: warn }

The local cache is worth enabling: once a key is known to be over its limit for the current window, the service answers from memory instead of touching Redis, which removes exactly the load spike an abusive client would otherwise create on your counter store.

Gotchas and edge cases

  • skip_if_absent: true silently disables the limit. With it set, a request missing the header emits no descriptor and is never limited — which is how “our limits stopped working” incidents begin. Prefer false, and pair it with an authentication filter that rejects unidentified requests earlier.
  • The window is fixed, not sliding. The reference service implements fixed windows per unit, so a client can spend a full budget at the end of one second and again at the start of the next. Where the boundary spike matters, layer a short-window descriptor (per second) under a longer one (per minute).
  • Descriptor cardinality lands in Redis. One key per distinct descriptor value per window. Limiting by raw IP on a public endpoint can create millions of short-lived keys; make sure the window unit and Redis memory policy account for it.
  • Domain name mismatches fail open. If the filter’s domain does not exist in the service config, every request matches nothing and passes. Assert the domain in a startup test.
  • Fail-open is invisible without a metric. Watch ratelimit.service.rate_limit.error and Envoy’s ratelimit.<domain>.error; a fail-open with no alert is an unlimited API that looks healthy.
  • Timeouts stack under load. A 1-second timeout with a struggling service adds a second to every request. Keep it in the tens of milliseconds and let the failure policy do its job.
  • Ingress and sidecar both limiting. Applying the same descriptors at two hops double-counts: the ingress consumes the budget, the sidecar consumes it again. Limit at one hop per descriptor, or use distinct route classes.
Failure policy when the rate limit service is unreachable With failure_mode_deny false the mesh admits all traffic and must alert on the error metric; with it true the mesh rejects everything and protects the upstream; a long timeout instead converts the outage into latency on every request. Rate limit service unavailable failure_mode_deny: false all traffic admitted no limits enforced requires an error alert default for read paths failure_mode_deny: true all traffic rejected upstream protected clients see 429 storms choose for costly writes timeout too generous every request waits latency becomes the outage worst of both modes keep it under 50 ms pick per route class: reads fail open, expensive writes fail closed

Verification and testing

Two checks prove the setup: that descriptors reach the service in the shape you expect, and that the fleet total is the configured number.

bash
# 1. Are descriptors arriving? The service exposes its decisions via statsd/prometheus.
kubectl port-forward deploy/ratelimit 6070:6070 &
curl -s localhost:6070/metrics | grep -E 'ratelimit_service_rate_limit_(over_limit|within_limit)'
# ratelimit_service_rate_limit_within_limit{key1="route_class_charges",key2="customer"} 4021
# ratelimit_service_rate_limit_over_limit{key1="route_class_charges",key2="customer"}   137
# An all-zero series means the descriptor never matched — check key order and names.

# 2. Fleet-wide ceiling with one key: accepted should equal the configured 50/s.
hey -z 30s -c 40 -H "X-API-Key: acct_loadtest" https://api.example.com/v1/charges | \
  grep -E '\[200\]|\[429\]'
# expect roughly 1500 x 200 over 30 s (50/s), the rest 429 — regardless of proxy count

If the accepted rate scales with the number of proxies, the requests are being handled by the local filter rather than the global one — the two filters can both be present, and the local one runs first.

Frequently Asked Questions

Should I use the local or the global rate limit filter?

Use the local filter to protect a single upstream from overload, where per-process counting is acceptable. Use the global filter for any limit you publish to clients, because only a shared counter makes the fleet total match the advertised number. Many meshes run both: a generous local guard plus a precise global limit.

Does the global filter add much latency?

One gRPC round trip to a service that performs one Redis operation, typically 0.5 to 2 ms within a cluster. Keep the filter timeout at 20 to 50 ms so a degraded service cannot dominate request latency, and enable the service's local cache so repeat rejections never reach Redis.

Why are my limits not being applied at all?

Almost always a descriptor mismatch: the domain name differs between filter and service config, the descriptor keys are in a different order, or skip_if_absent is dropping the descriptor when a header is missing. An unmatched descriptor is not an error — it simply has no limit — so check the service's per-descriptor counters first.

Can I express per-plan limits without redeploying?

Yes. The service reads its domain configuration from a runtime directory that can be backed by a ConfigMap or a mounted volume, so adding an override for one customer is a config change. For plan data that changes frequently, emit the plan as dynamic metadata from an authentication filter and use a metadata action so the descriptor carries the tier directly.