Edge & Gateway Rate Limit Enforcement

Every request your application rejects has already cost you a TLS handshake, a routing hop, and a worker thread — which is why the cheapest place to say “no” is the tier furthest from your business logic, a concern the Backend Middleware & Distributed Tracking reference treats as the outer ring of enforcement. Edge and gateway limiting moves the decision into Nginx, Envoy, an API gateway, or a CDN worker, where a rejection costs microseconds of proxy CPU instead of a full application request. The tradeoff is precision: the proxy tier knows the IP, the path, and maybe a header, but it does not know that this token belongs to an enterprise account with a negotiated 1,000 rps ceiling. This guide covers what each proxy layer can enforce, how its counter state is shared (or not), and where the boundary should sit between a coarse edge shield and a precise origin limiter.

The mental model to hold: the edge exists to absorb volume, the origin exists to enforce policy. A crawler hammering /v1/search with 40,000 requests a minute from twelve IPs is a volume problem — the edge should drop it without ever opening a connection to your service. A pro-tier customer exceeding their negotiated monthly quota is a policy problem, and no proxy has the account state to answer it correctly.

Mechanism: what each tier can actually count

A rate limiter needs three things: an identity to count against, a place to keep the count, and an atomic way to update it. What changes between tiers is where that state lives and how many machines share it.

Nginx limit_req keeps its counters in a shared memory zone local to one Nginx instance. The algorithm is a leaky bucket with an optional burst queue: requests arrive, drain at rate, and anything beyond burst is rejected with 503 (or 429 if you configure limit_req_status). State is O(1) per key with a fixed 128-byte slot per entry, so a 10 MB zone tracks roughly 80,000 distinct keys. Because the zone is per-instance, running four Nginx nodes behind a load balancer multiplies your effective limit by four — a limit of 100 rps becomes 400 rps in aggregate unless the balancer pins clients to nodes.

Envoy solves that with two distinct features. The local rate limit filter behaves like Nginx (per-process token bucket, no coordination). The global rate limit filter calls out to an external gRPC rate limit service that owns a shared Redis-backed counter, so every proxy in the mesh consults one truth. The cost is a network round trip on the hot path, typically 0.5–2 ms, and a hard decision about what happens when the service is unreachable.

API gateways (Kong, Tyk, Apigee, AWS API Gateway) sit one layer up and can usually resolve consumer identity — an API key, a JWT subject, a consumer group — before applying a policy. That makes them the first tier that can express “free tier gets 10 rps, pro gets 100 rps” without touching your application, provided you replicate plan data into the gateway’s configuration.

Edge workers (Cloudflare Workers, Fastly Compute, CloudFront Functions) run in hundreds of points of presence. Each PoP has its own state unless you use a coordinating primitive such as a Durable Object or a KV store with eventual consistency. Counting is therefore approximate by design: a client spread across ten PoPs sees roughly ten independent buckets, which is fine for blocking a 40,000 rps flood and useless for enforcing “exactly 100 requests per minute.”

Configuration reference: Nginx limit_req

The most widely deployed edge limiter is also the one most often misconfigured. Every knob and what it actually does:

The single most common production incident from this feature is omitting burst. A rate=10r/s limit with no burst rejects any two requests that arrive within the same 100 ms window — which a browser loading a page with six parallel XHRs will do every time. Start with burst=20 nodelay and tighten from measurements.

Implementation walkthrough: a two-tier setup

The pattern that survives production is a loose edge limit that only catches abuse, plus a tight origin limit that enforces the contract. Here is the edge half in Nginx, deliberately set well above any legitimate client’s rate:

# /etc/nginx/conf.d/ratelimit.conf
# Tier 1 (edge): volumetric shield. Generous, per-IP, catches floods only.
limit_req_zone $binary_remote_addr zone=perip:20m rate=50r/s;

# Tier 1b: a tighter zone for expensive endpoints, keyed by IP + route.
limit_req_zone $binary_remote_addr$uri zone=expensive:10m rate=5r/s;

limit_req_status 429;
limit_req_log_level warn;

server {
    listen 443 ssl http2;
    server_name api.example.com;

    # Burst absorbs legitimate parallelism; nodelay avoids adding latency.
    limit_req zone=perip burst=100 nodelay;

    location /v1/search {
        limit_req zone=expensive burst=10 nodelay;
        proxy_pass http://origin;
        # Pass the real client identity so the origin limiter counts correctly.
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Edge-Limited  $limit_req_status;
    }

    location / {
        proxy_pass http://origin;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

The origin half stays exactly as it would without an edge: an atomic per-key check against the Redis counter architecture, because only the application knows which plan the key belongs to.

-- Origin limiter (Redis EVAL): per-API-key token bucket, one round trip.
-- KEYS[1] = rl:{api_key}   ARGV = capacity, refill_per_sec, now_ms, cost
local cap  = tonumber(ARGV[1])
local rate = tonumber(ARGV[2])
local now  = tonumber(ARGV[3])
local cost = tonumber(ARGV[4])

local st     = redis.call('HMGET', KEYS[1], 'tokens', 'ts')
local tokens = tonumber(st[1]) or cap
local ts     = tonumber(st[2]) or now

-- Refill by elapsed time, never above capacity.
tokens = math.min(cap, tokens + (now - ts) / 1000 * rate)

local allowed = 0
if tokens >= cost then
  tokens  = tokens - cost
  allowed = 1
end

redis.call('HSET', KEYS[1], 'tokens', tokens, 'ts', now)
-- TTL = time to refill a full bucket, so idle keys evict themselves.
redis.call('PEXPIRE', KEYS[1], math.ceil(cap / rate * 1000) + 1000)

-- retry_after in whole seconds, minimum 1 when denied.
local retry = 0
if allowed == 0 then
  retry = math.max(1, math.ceil((cost - tokens) / rate))
end
return { allowed, math.floor(tokens), retry }

Set the edge limit at roughly five to ten times the tightest origin limit. If the edge is tighter than the origin, the origin’s headers become fiction: clients are told they have 80 requests remaining while a proxy silently rejects them, and no amount of correct client back-off logic can recover from that mismatch.

Distributed considerations: state that does not add up

Per-instance counters multiply with fleet size. This is not a subtle effect — it is the dominant source of “why is our 100 rps limit letting through 600 rps” incidents.

Two mitigations exist, and they are not equivalent. Dividing the configured limit by the instance count (100 / 6 ≈ 17r/s per node) keeps the fleet total roughly right but breaks the moment autoscaling changes the node count, and it punishes clients whose traffic lands unevenly. Moving to a shared counter — Envoy’s global service, a gateway plugin backed by Redis, or a Cloudflare Durable Object — costs a round trip but produces a number that matches what you documented. For anything customer-visible, pay the round trip. For a volumetric shield where “roughly 50 rps per node” is an acceptable description, keep it local and cheap.

Clock behaviour matters at the edge too: PoPs and proxy nodes drift, and a limiter whose window boundary depends on local time will reset at slightly different instants across the fleet. When exactness matters, derive time from the counter store (redis.call('TIME')) rather than from each proxy, exactly as the distributed algorithm sync guide describes for application limiters.

Failure modes and mitigations

• Shared limiter unreachable. Envoy’s failure_mode_deny and equivalent gateway settings decide whether an unreachable counter service means “allow everything” or “reject everything.” Default to fail-open for public read traffic and fail-closed for expensive write endpoints, and emit a distinct metric label so a silent fail-open is visible on a dashboard rather than discovered in a bill.
• Zone exhaustion. When an Nginx zone fills, new keys evict old ones and limits become porous. Size the zone for peak distinct keys, not peak requests, and alert on the limiting requests warning rate in the error log.
• Counting the proxy instead of the client. If the edge terminates and re-originates connections, $remote_addr at the next hop is the proxy’s IP, collapsing every client into one bucket. Use $http_x_forwarded_for with a trusted-proxy list, or the CDN’s client-IP header, and never trust a client-supplied header without validating the hop.
• Retry storms after a mass 429. Rejecting 20,000 requests at once teaches 20,000 clients to retry at the same moment. Always emit Retry-After at the edge, and jitter it slightly per client so the retry wave spreads out.
• Silent status mismatch. Nginx’s default 503 tells clients “server error, retry aggressively”; 429 tells them “you are limited, back off.” One directive separates a self-healing client from a retry storm.
• Edge tighter than origin. The origin’s X-RateLimit-Remaining becomes a lie. Keep a documented multiple between the tiers and assert it in a test that runs against staging.

Response contract at the edge

Whatever tier rejects a request, the response must look the same to a client. Standardise on 429 Too Many Requests, a Retry-After in delta-seconds, and — where the tier knows them — the X-RateLimit-* triplet described in rate-limit response headers. A useful convention is an internal marker header (X-RateLimit-Tier: edge|gateway|origin) stripped before the response leaves your network but visible in staging, so an engineer debugging an unexpected 429 can tell instantly which layer produced it. Never let one tier return 503 while another returns 429 for the same class of event; clients build their back-off logic on status codes and inconsistency defeats it.

Observability: knowing which tier said no

An edge limiter that rejects traffic silently is indistinguishable from an outage. The support ticket says “your API returns 429 for one of my keys,” and if the only evidence lives in a proxy access log rotated hourly, the investigation starts from nothing. Three signals make edge enforcement debuggable.

A decision counter per tier and outcome. Every layer that can reject increments a counter labelled with the tier, the route class, and the outcome. Nginx exposes this through the stub status module plus log-derived metrics; Envoy exports ratelimit.over_limit and ratelimit.error per descriptor; gateways expose plugin counters. Normalise them into one metric name so a single dashboard panel answers “who rejected this traffic.”

# Log the limiter verdict in a parseable field so a log pipeline can count it.
log_format ratelimited '$remote_addr $status $limit_req_status '
                       '"$request" key=$binary_remote_addr rt=$request_time';
access_log /var/log/nginx/api.log ratelimited;

$limit_req_status takes the values PASSED, DELAYED, REJECTED, DELAYED_DRY_RUN, and REJECTED_DRY_RUN. The dry-run values are the reason to always roll out with limit_req_dry_run on; first: you get exactly the rejection counts you would have produced, on production traffic, before any client is affected. Leave it on for a full traffic cycle — including the weekly batch job that only runs on Sunday night — and only then flip it off.

A rejection-rate alert with a tier label. The threshold that matters is not absolute 429 count but the share of requests rejected per tier. A jump from 0.2% to 14% at the edge means either an attack or a misconfiguration, and the two are distinguished by whether the rejections concentrate on a few keys. The alerting on 429 error rates guide covers the burn-rate formulation; the only edge-specific addition is the tier label so a paging alert names the layer.

Zone-pressure telemetry. For memory-zone limiters, track how full the zone is. Nginx logs a warning when it evicts, but by the time evictions are frequent your limits have quietly become unreliable. Alert on eviction warnings per minute, not on zone bytes, because the byte figure is not exported directly.

The pairing to avoid: an edge that rejects with no metric and an origin that emits perfect headers. Clients then see X-RateLimit-Remaining: 74 on their last successful response and a bare 429 on the next, with no Retry-After and no explanation. That combination generates more support load than the limiter saves in capacity.

Rollout sequence that does not page anyone

Enforcement changes are irreversible from the client’s perspective: a rejected request is gone. Sequence the rollout so every step is measurable and revertible.

• Measure first. Record the p50, p99, and max request rate per identity over a full week — including batch windows and mobile-app refresh storms.
• Set the candidate limit at 3–5× the observed p99 per identity, not at the average; the average tells you nothing about the client that legitimately bursts.
• Deploy with limit_req_dry_run on (or the gateway’s equivalent “log only” mode) and leave it for a full weekly cycle.
• Review the would-be rejections by identity. If a handful of keys dominate, decide whether each is abuse or a customer that needs a higher tier before you enforce.
• Enable enforcement on a single low-traffic route first, with limit_req_status 429 and a Retry-After in the response.
• Watch the rejection rate, upstream latency, and support queue for 24 hours; a healthy rollout shows rejections concentrated on a small number of identities.
• Widen to the remaining routes, then tighten the limit in steps no larger than 30% at a time.
• Document the final numbers publicly — clients cannot back off correctly against limits they cannot see.

The step engineers skip most often is the first. Setting an edge limit from intuition rather than from a measured distribution is how a legitimate customer’s nightly export job becomes an incident at 02:00, and the fix — raising the limit under pressure — teaches the organisation that limits are negotiable rather than designed.

Choosing a tier

• Traffic is anonymous, high-volume, and geographically spread → edge worker, approximate counting is fine.
• You run your own proxies and want cheap burst shaping → Nginx limit_req with generous burst and nodelay.
• You run a service mesh and need fleet-wide fairness per route → Envoy global rate limit service.
• Plans and consumers are already modelled in a gateway → gateway plugin, keyed by consumer.
• The limit depends on account state, billing, or per-request cost → application middleware; no proxy can do it correctly.

In this section

• Nginx limit_req vs an Application Limiter — when the shared-memory zone is enough and when it is not.
• Envoy Global Rate Limit Service Setup — descriptors, the gRPC service, and Redis-backed shared counters.
• Cloudflare Worker Rate Limiting at the Edge — per-PoP approximation versus Durable Object coordination.
• Kong and API Gateway Rate Limit Plugins — consumer-scoped policies and their storage backends.
• Layering Edge and Origin Rate Limits — picking the multiple between tiers so headers stay honest.

Related

• Backend Middleware & Distributed Tracking — the parent reference for where enforcement lives in the request path.
• Redis Counter Architecture — the shared store behind gateway plugins and origin limiters alike.
• Tiered Access & Quota Enforcement — the policy layer that proxies cannot express.
• Leaky Bucket Mechanics — the algorithm limit_req implements under the hood.
• Rate-Limit Response Headers — keeping the contract identical across tiers.