Unit Testing Rate Limit Middleware

Limiter tests get a reputation for flakiness because the first instinct is to sleep until a window rolls, which makes a suite slow, timing-dependent, and unreliable on a busy runner. Injecting the clock removes all three problems at once and turns hours of manual verification into a few milliseconds of assertions — the fast layer of the strategy in rate limit testing and validation.

The problem in concrete numbers

Testing a 100-per-hour limit by waiting for the window to roll costs an hour per case. With ten boundary cases that is a suite nobody runs. Worse, each case that does run is fragile: a garbage-collection pause of 200 ms on a shared CI runner shifts a request across a window boundary and the assertion flips.

With an injected clock, the same ten cases run in under five milliseconds, deterministically, on any machine — and they can cover things wall-clock tests cannot, like a clock stepping backwards after an NTP correction.

Wall-clock tests against an injected clock Wall-clock tests take an hour per window case and fail intermittently under runner load, while an injected clock runs the same cases in milliseconds with identical results every time and can simulate clock steps. Same ten cases, two test designs real clock and sleeps runtime: hours flaky under runner load cannot test clock steps gets disabled within a month injected clock runtime: milliseconds identical on every machine clock steps are just inputs runs on every commit the design constraint: time must be a parameter, never a hidden global

What to separate before testing

Layer Responsibility Test style Needs a store?
Decision function State + now + config → verdict Table-driven and property tests No
Store adapter Read, apply, write atomically Integration, real store Yes
Middleware Identity, headers, status, next() Unit with a stubbed limiter No
Configuration Plan → limit resolution Unit No

Fusing the decision function into the store adapter is the design mistake that makes limiters untestable. Keep the arithmetic pure and the I/O thin, and 90% of your cases need neither a container nor a clock that moves on its own.

Step-by-step: building the suite

typescript
// limiter.ts — pure decision, no I/O, no hidden clock.
export interface Config { capacity: number; refillPerSec: number; }
export interface State { tokens: number; ts: number; }
export interface Verdict {
  allowed: boolean; remaining: number; retryAfterMs: number; resetMs: number; state: State;
}

export function decide(prev: State | null, now: number, cfg: Config, cost = 1): Verdict {
  const s = prev ?? { tokens: cfg.capacity, ts: now };
  // Clamp elapsed at zero: an NTP correction must never produce a negative refill.
  const elapsed = Math.max(0, now - s.ts);
  const tokens = Math.min(cfg.capacity, s.tokens + (elapsed / 1000) * cfg.refillPerSec);
  const resetMs = Math.ceil(((cfg.capacity - tokens) / cfg.refillPerSec) * 1000);

  if (tokens >= cost) {
    return { allowed: true, remaining: Math.floor(tokens - cost), retryAfterMs: 0,
             resetMs, state: { tokens: tokens - cost, ts: now } };
  }
  return { allowed: false, remaining: 0,
           retryAfterMs: Math.ceil(((cost - tokens) / cfg.refillPerSec) * 1000),
           resetMs, state: { tokens, ts: now } };   // denial does not consume
}
typescript
// limiter.test.ts — table-driven boundaries plus the adversarial cases.
import { describe, it, expect } from "vitest";
import { decide, State, Config } from "./limiter";

const CFG: Config = { capacity: 10, refillPerSec: 1 };   // 10 burst, 1/s sustained

function run(times: number[], cfg = CFG, cost = 1) {
  let s: State | null = null;
  return times.map((t) => { const v = decide(s, t, cfg, cost); s = v.state; return v; });
}
const admitted = (vs: ReturnType<typeof run>) => vs.filter((v) => v.allowed).length;

describe("admission", () => {
  const cases: Array<[string, number[], number]> = [
    ["cold start admits capacity",        Array(10).fill(0),                        10],
    ["overflow is clipped",               Array(15).fill(0),                        10],
    ["boundary at exactly one second",    [...Array(10).fill(0), 1000],             11],
    ["steady pacing sustains the rate",   Array.from({ length: 20 }, (_, i) => i * 1000), 20],
    ["idle then burst refills to cap",    [...Array(10).fill(0), ...Array(10).fill(60_000)], 20],
    ["clock step backwards is harmless",  [0, 0, 0, -5_000, -5_000],                5],
  ];
  it.each(cases)("%s", (_name, times, expected) => {
    expect(admitted(run(times))).toBe(expected);
  });
});

describe("denial semantics", () => {
  it("never advertises a zero wait", () => {
    const denied = run(Array(12).fill(0)).filter((v) => !v.allowed);
    expect(denied.every((v) => v.retryAfterMs > 0)).toBe(true);
  });

  it("does not extend the wait under a retry loop", () => {
    const vs = run([...Array(11).fill(0), 100, 200, 300, 400, 500]);
    const waits = vs.filter((v) => !v.allowed).map((v) => v.retryAfterMs);
    // Each retry happens later in real time, so the reported wait must shrink.
    expect(waits.every((w, i) => i === 0 || w <= waits[i - 1])).toBe(true);
  });

  it("rejects a cost larger than capacity without draining the bucket", () => {
    const v = run([0], CFG, 25)[0];
    expect(v.allowed).toBe(false);
    expect(v.state.tokens).toBe(CFG.capacity);      // untouched
  });
});

describe("property: admissions are bounded", () => {
  it("never exceeds elapsed x rate + capacity for random traces", () => {
    for (let seed = 0; seed < 200; seed += 1) {
      let t = 0;
      const times = Array.from({ length: 300 }, () => (t += (seed * 7 + 13) % 250));
      const n = admitted(run(times));
      const bound = Math.ceil((t / 1000) * CFG.refillPerSec) + CFG.capacity;
      expect(n).toBeLessThanOrEqual(bound);
    }
  });
});
The six cases that catch nearly every limiter bug Cold start, overflow, the window boundary, steady pacing, idle-then-burst, and a backwards clock step each map to a distinct implementation mistake, from uninitialised buckets to negative refill. Six inputs, six distinct bugs cold start bucket not initialised to full capacity overflow off-by-one at the capacity comparison boundary strict versus inclusive time comparison steady pacing refill rate off by the period conversion idle then burst refill not clamped to capacity clock steps back negative elapsed gives an unexplained lockout

Testing the middleware itself

The middleware layer has its own bugs, and none of them involve arithmetic: a missing header, a next() called twice, an identity resolved from the wrong place. Stub the limiter and assert on the HTTP surface.

typescript
// middleware.test.ts — the limiter is stubbed; only the HTTP behaviour is under test.
import { describe, it, expect, vi } from "vitest";
import { rateLimit } from "./middleware";

const res = () => {
  const headers: Record<string, string> = {};
  return {
    headers, statusCode: 200,
    set: (k: string, v: string) => { headers[k.toLowerCase()] = v; },
    status(code: number) { this.statusCode = code; return this; },
    json: vi.fn(),
  };
};

it("sets the full triplet on an allowed request and calls next once", async () => {
  const next = vi.fn();
  const r = res();
  await rateLimit({ consume: async () => ({ allowed: true, remaining: 7, retryAfterMs: 0, resetMs: 3000 }) })(
    { header: () => "acct_1", ip: "10.0.0.1" } as any, r as any, next);
  expect(next).toHaveBeenCalledTimes(1);
  expect(r.headers["x-ratelimit-remaining"]).toBe("7");
  expect(r.headers["retry-after"]).toBeUndefined();       // only on a denial
});

it("returns 429 with Retry-After and does not call next", async () => {
  const next = vi.fn();
  const r = res();
  await rateLimit({ consume: async () => ({ allowed: false, remaining: 0, retryAfterMs: 1400, resetMs: 1400 }) })(
    { header: () => "acct_1", ip: "10.0.0.1" } as any, r as any, next);
  expect(next).not.toHaveBeenCalled();
  expect(r.statusCode).toBe(429);
  expect(Number(r.headers["retry-after"])).toBeGreaterThanOrEqual(1);   // never zero
});

it("fails open when the store throws, and marks the response", async () => {
  const next = vi.fn();
  const r = res();
  await rateLimit({ consume: async () => { throw new Error("ECONNREFUSED"); } })(
    { header: () => "acct_1", ip: "10.0.0.1" } as any, r as any, next);
  expect(next).toHaveBeenCalledTimes(1);                  // configured fail-open
  expect(r.headers["x-ratelimit-degraded"]).toBe("1");    // and it is observable
});
Which layer each assertion belongs to Arithmetic and boundary behaviour belong to the pure decision function, headers and status codes to the middleware with a stubbed limiter, and atomicity and key expiry to an integration test against a real store. Put each assertion where it can be fast and certain decision function refill and boundaries retry-after arithmetic clock steps, cost limits middleware status codes header presence next() called once integration atomicity under load key expiry script loading a mocked store can “prove” a non-atomic script atomic — keep that assertion out of unit tests

Gotchas and edge cases

  • Date.now() inside the algorithm. One call is enough to make every window test dependent on wall time. Pass now in, always.
  • Asserting on booleans only. A test that checks allowed === false passes even when retryAfterMs is nonsense. Assert the numbers clients will actually use.
  • Shared state between tests. A module-level bucket map leaks between cases and produces order-dependent failures. Construct fresh state per test.
  • Testing the store adapter with mocks. A mocked Redis will happily “prove” that a non-atomic script is atomic. Atomicity belongs to an integration test against a real store.
  • Floating-point equality. expect(tokens).toBe(0.30000000000000004) fails on a different platform. Assert on the integer remaining, or compare with a tolerance.
  • Forgetting next() semantics. A middleware that calls next() and writes a 429 produces a response with two bodies and a hard-to-trace error in production.

Verification and testing

bash
# The whole suite should be fast enough to run on every save.
npx vitest run limiter.test.ts middleware.test.ts --reporter=basic
# ✓ limiter.test.ts (12 tests) 4ms
# ✓ middleware.test.ts (3 tests) 6ms
# Test Files  2 passed (2)
#      Tests  15 passed (15)
#   Duration  118ms

# Coverage should reach the decision function completely; the adapter is covered elsewhere.
npx vitest run --coverage --coverage.include='src/limiter.ts'

If the suite takes more than a second, something is sleeping or talking to a network. Find it and inject a clock or a stub — a limiter suite that is slow enough to skip is a limiter suite that will be skipped.

Frequently Asked Questions

How do I test a window rollover without waiting?

Pass the timestamp as a parameter and call the decision function with the times you want. A rollover is just two calls with timestamps either side of the boundary, which runs in microseconds and gives the same answer on every machine.

Should unit tests use a real Redis?

No — the decision logic should not touch a store at all. Use a real store for a small number of integration tests that specifically verify atomicity, key expiry, and script loading, and keep the arithmetic tests pure so they stay fast and deterministic.

What is worth property testing?

The invariant that admissions over any interval never exceed elapsed time times the rate, plus the burst. Random arrival traces explore boundary interactions a hand-written table will miss, and the bound is easy to state and cheap to check.

How do I test the fail-open path?

Stub the limiter to throw and assert both halves of the contract: the request proceeds, and the degraded state is observable — a header, a metric, or a log field. A fail-open that leaves no trace is untestable in production and invisible during an incident.