Layering Edge and Origin Rate Limits

Two limiters in the same request path is the normal production shape, and the only hard rule is that they must never disagree about who is in charge. Getting the ratio right is the whole job: too close together and clients receive rejections that contradict the quota headers they were just given; too far apart and the edge stops shielding anything. This page turns the tier model from edge and gateway enforcement into a specific number you can configure and test.

The problem in concrete numbers

Your API publishes 100 requests per minute per key. Your edge shield is per-IP because the edge cannot authenticate. A single office of 40 developers shares one outbound address, so their combined legitimate traffic is 4,000 requests per minute from one IP — forty times the per-key limit, and entirely legitimate. Set the edge at 200 per minute per IP to “match” the published limit and you have just blocked a customer’s entire engineering team while their per-key headers cheerfully report 87 remaining.

The inverse mistake is subtler. Set the edge at 100,000 per minute and it will never engage before your origin pods have already spent CPU rejecting the flood. The shield exists to be hit occasionally — by attackers, not by customers.

Deriving the ratio

The edge limit is not a multiple of the published limit by aesthetic preference — it is derived from how many identities share an address, and how much burst each of them legitimately produces.

edge_limit_per_ip ≈ published_limit × max_keys_per_ip × burst_factor

Run the measurement before choosing. A public API with mostly individual developers might see max_keys_per_ip = 3; one selling to enterprises regularly sees 200 keys behind a single corporate egress address, and any edge limit derived from a smaller assumption will page you the day that customer onboards.

Where the edge can authenticate — a gateway that resolves a consumer, or a worker that validates a signed token — the calculation collapses: use the same identity as the origin and set the edge limit at 2–3× the published number purely to absorb retries and clock differences.

Step-by-step: layering without contradictions

• Publish exactly one number per plan, and enforce it at the origin where account state lives.
• Measure keys-per-address at the 99th percentile over at least one full week.
• Compute the edge limit from the formula and round up to a memorable number.
• Give both tiers the same status code (429) and the same Retry-After unit (delta-seconds).
• Mark the tier in the rejection so support and dashboards can attribute it — a header, a log field, or both.
• Only the origin emits X-RateLimit-*, because only it knows the published quota.
• Add a monitor comparing edge rejection rate to origin rejection rate; edge should be a small fraction under normal traffic.
• Assert the ordering in CI: a synthetic client at just under the published limit must never receive an edge rejection.

# Edge tier: derived, not guessed. 100/min published x 20 keys/IP x 2 burst = 4000/min.
limit_req_zone $binary_remote_addr zone=shield:32m rate=67r/s;   # ~4000/min
limit_req_status 429;

server {
    listen 443 ssl http2;
    server_name api.example.com;
    limit_req zone=shield burst=200 nodelay;

    location /v1/ {
        proxy_pass http://origin;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

// Origin tier: the published contract, and the only tier that emits quota headers.
app.use(async (req, res, next) => {
  const key = req.apiKey;                       // set by the auth middleware
  const plan = await plans.get(key);            // free: 100/min, pro: 1000/min
  const v = await limiter.consume(`rl:${key}`, plan.perMinute, 60);

  res.set("X-RateLimit-Limit", String(plan.perMinute));
  res.set("X-RateLimit-Remaining", String(v.remaining));
  res.set("X-RateLimit-Reset", String(v.resetSeconds));
  res.set("X-RateLimit-Tier", "origin");        // strip at the public boundary if you prefer

  if (v.allowed) return next();
  res.set("Retry-After", String(Math.max(1, v.resetSeconds)));
  res.status(429).json({ error: "rate_limited", retry_after: v.resetSeconds });
});

What each tier must not do

That last line is the test for a healthy layering: the origin limiter must be correct on its own. An architecture where the published limit is only correct because the edge is also counting has two single points of failure and no way to reason about either.

Gotchas and edge cases

• Both tiers emitting quota headers. Clients then see two contradictory X-RateLimit-Remaining values depending on which layer answered. Only the tier that owns the published number should emit them.
• Retry storms crossing tiers. A mass edge rejection sends thousands of clients into synchronised back-off; when they return together the origin absorbs the spike. Jitter Retry-After at the edge by a few seconds per client.
• Different units. One tier emitting Retry-After in seconds and the other in an HTTP date is legal and awful. Standardise on delta-seconds; see parsing Retry-After for what clients must otherwise handle.
• Health checks caught by the shield. Monitoring traffic from a single address is exactly the pattern an IP limiter punishes. Exempt monitors by path or shared secret.
• Edge limit tuned for today’s customer mix. The ratio drifts as enterprise customers arrive. Re-derive it quarterly from fresh keys-per-address data.
• Silent divergence after an autoscale. Per-node edge limits multiply with node count; a scale-up quietly loosens the shield. Prefer a shared edge counter, or recompute the per-node number as part of the scaling policy.

Verification and testing

# A synthetic client just under the published limit must NEVER see an edge rejection.
# 95 requests per minute against a 100/min published limit, from one address.
for i in $(seq 1 95); do
  curl -s -o /dev/null -w "%{http_code} %{header_json}\n" \
    -H "X-API-Key: layer_test" https://api.example.com/v1/ping
  sleep 0.6
done | grep -c '"x-ratelimit-remaining"'
# expect 95 — every response carries origin headers, so the edge never intervened

# Confirm the shield still bites when it should: 200 requests in 2 seconds.
seq 1 200 | xargs -P50 -I{} curl -s -o /dev/null -w "%{http_code}\n" \
  https://api.example.com/v1/ping | sort | uniq -c
# expect a mix of 200 and 429; the 429s should lack X-RateLimit headers (edge origin)

Automate the first check. It encodes the invariant that matters — a compliant client never meets the shield — and it will fail loudly the day someone tightens the edge limit without recomputing the ratio.

Frequently Asked Questions

Related

• Edge & Gateway Enforcement — the parent guide on tier responsibilities.
• Nginx limit_req vs an Application Limiter — configuring the shield tier itself.
• Cloudflare Worker Rate Limiting at the Edge — approximate counting far from the origin.
• Rate-Limit Response Headers — the contract that only one tier should own.