Post

1/ Ever watched your servers start sweating because a single rogue bot or a buggy client retry loop went wild and hammered your API with hundreds of requests per second? That’s rate limiting in action – the invisible guardrail that keeps your system alive, fair, and costs in check.

2/ There are exactly 5 algorithms you must know. I’ll give you the mental model, when to pick each, and the real trade-offs (the stuff interviewers actually test).

3/ 1. Token Bucket Tokens refill at a steady rate into a bucket. Each request grabs one. Run out? Block or queue. Superpower: Allows natural bursts (uploading 20 photos at once feels smooth). Real-world: Stripe, GitHub, AWS APIs. Pick this when your users are humans, not robots.

4/ 2. Leaky Bucket Requests pour in but only “leak out” to your servers at a fixed constant rate. Bucket fills up? Extra requests get dropped. Superpower: Perfectly smooth downstream traffic – your DB never sees spikes. Catch: Zero bursts allowed. Use when stability > flexibility.

5/ 3. Fixed Window Time sliced into rigid blocks (every 60 seconds). Count inside the block, reset sharp at the boundary. Superpower: Dead simple, almost zero memory. Big catch: Everyone games it – spam at 11:59 + 12:01 and you sneak double the limit.

6/ 4. Sliding Window Log Store a timestamp for every single req. On new req, drop anything older than the window & count the rest. Superpower: Most accurate – no boundary tricks possible. Catch: Memory explodes at scale (millions of timestamps = pain) Only when absolute fairness is non-negotiable.

7/ 5. Sliding Window Counter (Big Tech’s default) Hybrid genius. Keep counters for previous + current window + smart weighted math. Example: 100 req/min limit, 30s into the minute. Previous window: 80 Current: 20 Weighted = (80 × 0.5) + 20 = 60 → still under limit.

Sliding Window Counter is accurate enough for production, low memory, no timestamp spam.

8/ At real scale, in-memory counters are useless. One user round-robins across 10 pods and cheats the limit. Fix: Central Redis (or equivalent). Store counters there. Wrap “check + increment” in Lua scripts → atomic, one network hop, zero races even at 10k+ RPS.

9/ Placement is the real engineering decision: • Edge/CDN → kill junk before it hits your VPC • API Gateway → cleanest for most teams • Server-side → only when you need deep per-user business logic Layer them. That’s how you think like someone who actually ships.

10/ Resilience Redis dies ? Fail-open (keep UX alive with circuit breaker) or Fail-closed (429 everyone to protect the farm) ? I bias open. Always return these headers: X-RateLimit-Remaining X-RateLimit-Reset Retry-After Your frontend team will love you.

11/ Tips • Dynamic limits per tier ? Pull from Consul/etcd. • Log every 429 → spot abuse patterns early. • Test race conditions and window boundaries like your on-call depends on it (it does). • Real war story: badly tuned rate limiters have taken down major services during flash sales.