Redis Counter Architecture for Distributed API Throttling

ratelimit:{service}:{client_id}:{window_epoch}

# Safe fixed-window initialization: set TTL only on key creation
# In practice, wrap both calls in a Lua script or use the conditional below
INCR ratelimit:api-gw:client_8f3a:1715000000
# Application-side: if result == 1: EXPIRE ratelimit:... 60

local n = redis.call('INCR', KEYS[1])
if n == 1 then redis.call('EXPIRE', KEYS[1], ARGV[1]) end
return n

import { Request, Response, NextFunction } from 'express';
import Redis from 'ioredis';

const redis = new Redis.Cluster([{ host: 'redis-node-1', port: 6379 }], {
 maxRetriesPerRequest: 1,
 retryStrategy: (times) => Math.min(times * 50, 2000),
 enableAutoPipelining: true
});

export const rateLimitMiddleware = async (req: Request, res: Response, next: NextFunction) => {
 const clientId = req.headers['x-client-id'] || req.ip;
 const windowKey = `ratelimit:api:${clientId}:${Math.floor(Date.now() / 60000)}`;
 const limit = 100;

 try {
 // Use a Lua script to atomically increment and set TTL only on first request
 const currentCount = await redis.eval(
 "local n = redis.call('INCR', KEYS[1])\nif n == 1 then redis.call('EXPIRE', KEYS[1], ARGV[1]) end\nreturn n",
 1, windowKey, '60'
 ) as number;

 const remaining = Math.max(0, limit - currentCount);
 res.set({
 'X-RateLimit-Limit': String(limit),
 'X-RateLimit-Remaining': String(remaining),
 'X-RateLimit-Reset': String(Math.ceil(Date.now() / 60000) * 60000)
 });

 if (currentCount > limit) {
 return res.status(429).json({ error: 'Rate limit exceeded' });
 }
 next();
 } catch (err) {
 // Fail-open strategy: allow request but log Redis degradation
 console.error('Redis counter sync failed:', err);
 next();
 }
};

// app.use(rateLimitMiddleware);

from fastapi import FastAPI, Depends, HTTPException, Request, Response
from redis.asyncio import Redis
from typing import Annotated
import time

app = FastAPI()


async def get_redis() -> Redis:
    return Redis(host="redis-cluster-1", port=6379, decode_responses=True)


RedisClient = Annotated[Redis, Depends(get_redis)]


async def rate_limit_dependency(
    request: Request,
    response: Response,
    redis: RedisClient,
    limit: int = 100,
    window: int = 60,
):
    client_id = request.headers.get("x-client-id", request.client.host)
    window_epoch = int(time.time()) // window
    key = f"ratelimit:api:{client_id}:{window_epoch}"

    try:
        # Use a Lua script to atomically increment and set TTL only on first call
        lua = "local n=redis.call('INCR',KEYS[1]) if n==1 then redis.call('EXPIRE',KEYS[1],ARGV[1]) end return n"
        current = await redis.eval(lua, 1, key, window)
    except Exception:
        return  # Fail-open on Redis outage

    remaining = max(0, limit - current)
    response.headers["X-RateLimit-Limit"] = str(limit)
    response.headers["X-RateLimit-Remaining"] = str(remaining)
    response.headers["X-RateLimit-Reset"] = str((window_epoch + 1) * window)

    if current > limit:
        raise HTTPException(status_code=429, detail="Rate limit exceeded")


@app.get("/api/v1/data")
async def get_data(_=Depends(rate_limit_dependency)):
    return {"status": "ok"}

-- KEYS[1] = rate limit key
-- ARGV[1] = window size (seconds)
-- ARGV[2] = max requests
-- ARGV[3] = current timestamp

local key = KEYS[1]
local window = tonumber(ARGV[1])
local limit = tonumber(ARGV[2])
local now = tonumber(ARGV[3])

-- Remove expired entries
redis.call('ZREMRANGEBYSCORE', key, 0, now - window)

-- Count current requests
local current = redis.call('ZCARD', key)

if current < limit then
 redis.call('ZADD', key, now, now .. '-' .. math.random(100000))
 redis.call('EXPIRE', key, window)
 return 0 -- Allowed
else
 return 1 -- Denied
end

ratelimit:{client_8f3a}:window_1715000000

The {client_8f3a} tag ensures Redis hashes the key based on the client identifier, routing all related counters to the same primary node. This eliminates CROSSSLOT errors during pipeline execution.

Failover routing requires middleware to implement retry logic with exponential backoff. During primary node promotion, counters may experience brief unavailability. Platform teams should configure cluster-require-full-coverage no to allow partial cluster operation, accepting temporary quota inconsistencies in favor of system availability. Cross-node synchronization is handled natively via Redis Cluster’s gossip protocol and asynchronous replication, though eventual consistency means brief counter drift (<50ms) is acceptable in distributed rate limiting.

Edge & Reverse Proxy Interception

Offloading counter logic to the network layer reduces backend latency and standardizes 429 response headers. For edge-level enforcement, teams frequently configure NGINX’s limit_req_zone / limit_req directives or the Kong rate-limiting plugin to offload counting logic before requests reach application servers.

Reverse proxies can integrate with Redis via modules like ngx_http_redis_module or envoy-ratelimit. The architecture follows:

1. Proxy extracts client identifier from JWT, API key, or IP.
2. Proxy executes atomic Lua script against Redis cluster.
3. Proxy injects X-RateLimit-* headers into upstream requests.
4. If limit exceeded, proxy returns 429 Too Many Requests immediately, bypassing application servers.

Shared memory vs Redis-backed state depends on deployment scale. Single-node proxies use limit_req_zone with shared_memory, while distributed edge networks require centralized Redis backing to maintain consistent quotas across multiple ingress points. Header injection must propagate downstream so microservices can adjust internal processing priorities based on remaining quota.

Client-Side Interceptors & Telemetry Workflows

Client-side interceptors must parse X-RateLimit headers, implement jittered backoff algorithms, and propagate trace IDs to correlate throttling events with backend counter increments.

Production frontend/SDK implementation patterns:

• Header Parsing: Extract X-RateLimit-Remaining and X-RateLimit-Reset to preemptively throttle outbound requests.
• Exponential Backoff with Jitter: delay = min(cap, base * 2^attempt) + random(0, jitter) prevents thundering herd during quota resets.
• Retry Budgets: Maintain a sliding window of allowed retries (e.g., 10% of total requests) to prevent infinite retry loops during sustained 429 responses.
• OpenTelemetry Integration: Attach traceparent headers and custom attributes (http.response.status_code=429, rate_limit.remaining=0) to distributed traces. This enables correlation between client retry spikes and backend Redis counter increments.

Platform teams should expose telemetry dashboards tracking rate_limit.exceeded_total, rate_limit.drift_seconds, and redis.pipeline_latency_p99 to monitor enforcement accuracy.

Performance Benchmarking & Production Tuning

Production readiness requires continuous benchmarking of P99 latency, pipeline batching efficiency, and memory eviction thresholds to ensure counter accuracy under sustained DDoS or flash-crowd conditions.

Key tuning parameters:

• Connection Multiplexing: Use ioredis enableAutoPipelining or redis-py pipeline(transaction=False) to batch counter increments. Reduces round-trip latency by 60-80%.
• Eviction Policies: Configure maxmemory-policy volatile-ttl to prioritize expiration of rate limit keys over persistent session data.
• Memory Footprint: Monitor used_memory_peak and mem_fragmentation_ratio. Sliding window logs should be capped at ZREMRANGEBYRANK during background maintenance.
• Latency Profiling: Deploy redis-cli --latency-history and APM integration to track COMMAND execution time. P99 counter increments should remain <2ms in cluster deployments.
• Counter Drift Monitoring: Implement periodic reconciliation jobs that compare expected vs actual counts. Drift >0.5% indicates pipeline failures or clock skew in distributed environments.

By adhering to these architectural principles, engineering teams deploy resilient, scalable rate limiting systems that maintain strict quota enforcement while preserving system availability under extreme load conditions.

For the specific tradeoff between issuing INCR+EXPIRE as separate commands versus wrapping the whole decision in a Lua script, see Redis Lua vs INCR Rate Limiting, which walks through the lost-EXPIRE race in detail.

Connection Management Under Load

The limiter is on the path of every request, so its connection behaviour matters more than that of any other store client in the application.

Pool size should be small and fixed. Each application process needs only a handful of connections, because commands are short and pipelined; a pool of fifty per process across two hundred processes exhausts the store’s connection limit long before its CPU. Size the pool from measured concurrency, not from optimism.

Timeouts must be explicit and short. Both a connect timeout and a command timeout are needed: many clients set only the former, so a connected but stalled store blocks indefinitely. Tens of milliseconds is right for a same-zone store — enough for a normal round trip and jitter, far below any request budget.

Offline queueing should be disabled. Clients that queue commands during an outage appear to succeed, then deliver a flood the moment the store returns, producing a second spike immediately after recovery. Failing fast lets the degradation policy run instead.

Circuit breaking turns a long outage into a cheap one. After a run of failures, stop calling the store for a cooldown and apply the configured policy immediately; probe with a single request to close the breaker. Without a breaker, every request pays the timeout for the whole outage, which is how a store problem becomes a latency problem for endpoints that have no limiter at all.

Sharing the Store, or Not

The strong recommendation is a dedicated instance or at minimum a dedicated database index for limiter state, and the reasons are concrete rather than aesthetic.

A cache and a limiter want opposite eviction behaviour. A cache is configured to discard whatever is least useful when memory runs short; a limiter’s keys are all useful, and discarding one grants a client a fresh allowance at exactly the moment the system is under pressure. Sharing an instance means the cache’s configuration silently governs the limiter’s correctness.

Workload shape differs too. Cache traffic is read-heavy with large values; limiter traffic is small, write-heavy, and latency-sensitive. Mixing them means a large cache read can sit in front of a limiter command in the same connection’s queue, adding latency that appears random.

Finally, blast radius. A limiter that shares an instance with session storage takes sessions down when it exhausts memory, and vice versa. Separate databases on one instance is a workable compromise for small deployments; separate instances is the answer once the limiter is on the critical path for meaningful traffic.

Capacity Planning for the Counter Store

Three numbers size the store, and all three are measurable before launch.

Active keys — distinct identities seen within one key lifetime, not total customers. With correct expiries, an application with a million monthly customers and fifty thousand active in any hour holds fifty thousand keys, not a million.

Bytes per key — the value plus the key name plus per-key overhead, which for small values is dominated by the last two. Measure it rather than estimating: a hundred bytes per key is a reasonable planning figure for a compact schedule, and several times that for a hash with multiple fields.

Commands per second — one per request in the simple case, more if you enforce several windows or check a quota separately. This is the number that decides whether one instance suffices, and it is worth measuring against your actual command mix rather than against a published benchmark.

Alert on all three. Key count and memory growing faster than customer growth means an expiry is missing somewhere; command rate growing faster than request rate means somebody added a second check to the hot path.

Treated as infrastructure rather than as a library detail — dedicated, bounded, timed out, and measured — the counter store disappears from your incident reports entirely, which is the only review a piece of shared infrastructure ever really gets.

Choosing Data Structures for the Counter

Redis offers several shapes for limiter state, and the choice affects both memory and the operations available.

A string holding a single number is the cheapest option and suits window counters and schedule timestamps: one key, one value, atomic increment or set, minimal overhead. It cannot express multi-field state, which is exactly why algorithms that need only one number should prefer it.

A hash holds several fields under one key and suits token buckets, which need a count and a timestamp. It costs slightly more per key and keeps related fields together so one script can read and write them atomically.

A sorted set holds timestamped entries and is the natural home for an exact log, with range operations for pruning. It is by far the most expensive per key, and its cost scales with request volume rather than with client count, which is the property that makes exact logs a specialist choice.

The general rule: pick the smallest structure that expresses the state your algorithm needs, and never spread one decision’s state across two keys. Two keys mean two operations, which means either a race or a multi-key script that will not run on a clustered store unless both keys hash together.

A closing note on evolution: the counter store outlives most of the code around it. Version the key prefix from day one, keep the state shape small enough to reason about, and document what each key holds. Those three decisions cost nothing now and are what make it possible to change algorithms, split shards, or migrate instances later without a coordinated rewrite of every limiter in the system.

Reviewing the Limiter Periodically

Limits age. Traffic patterns shift as customers change how they integrate, plans are added, endpoints get faster or slower, and the fleet grows. A limiter configured correctly two years ago is rarely configured correctly today, and nothing in the system will point that out.

A short quarterly review keeps it honest. Re-derive the burst allowance from a fresh week of arrival timestamps, because the interaction sizes that justified the original number will have changed. Re-measure the accepted ceiling with a single-key load test against the current deployment, because node counts and store topology drift. Compare the configured numbers against the published ones and against whatever the gateway believes, since all three tend to diverge quietly. And read the rejection distribution: if it has moved from concentrated on a few identities to spread across many, a limit that used to constrain abuse is now constraining customers.

The review takes an hour and usually produces one small change — a burst raised, an exemption removed, a limit that was never enforced because a configuration alias moved. Skipping it produces the opposite pattern: no changes for two years, then an urgent one during an incident, made without measurement and defended forever afterwards.

Related

• Backend Middleware & Distributed Tracking — the parent topic covering middleware placement and framework integration.
• Redis Lua vs INCR Rate Limiting — when atomic Lua is mandatory and when plain INCR is enough.
• Express.js Rate Limit Middleware — wiring this Redis store into a Node.js request pipeline.
• Distributed Algorithm Sync — clock skew and cross-node consistency behind Redis replication.