One client stuck in a retry loop with no backoff can put more load on your API in a minute than your other ten thousand clients combined. From your side it won’t look like an attack. It looks like a bug on their end, and it becomes an incident on yours. A rate limiter caps that client before it gets there.

What does a rate limiter actually decide?

On every request, the limiter asks one thing: has this key made too many requests in the current window? Three choices turn that into something you can implement.

The key is what you count against: an IP address, an API key, an authenticated user ID. Pick the wrong one and you either block innocent users who share an identity, or let the real abuser hide among them.

The window and the limit only mean something together. “100 requests” is not a limit; 100 per second and 100 per day are two very different APIs.

The action is what happens when the key is over. Most APIs reject with 429 Too Many Requests. A background job or a webhook sender is more likely to queue the work and drain it later, which is the job a message queue does (see What Is a Message Queue and How to Use One).

Which algorithm, which store, which headers: all of that is implementation around this one check.

Four rate limiting algorithms and what each gets wrong

Fixed window counter. Increment a counter for the key and reset it when the clock crosses a boundary, say every minute on the minute. It costs one integer per key. The flaw: a client that sends 100 requests at 0:59 and another 100 at 1:00 has sent 200 requests in two seconds without breaking “100 per minute” in either window. At the boundary, a fixed window lets through up to double the stated rate.

Sliding window log. Store the timestamp of every request for the key (a Redis sorted set fits), and on each new request count how many fall within the last 60 seconds. It’s exact, so the boundary trick doesn’t work. The cost is memory that grows with traffic: counting a million timestamps per key is not the same as reading one integer, the same gap as scanning a list versus reading an index that Big O Notation Explained With Code Examples puts a name to.

Sliding window counter. This is the one most APIs should use. Keep two fixed-window counters, the current and the previous, and estimate the sliding count as current_count + previous_count * (overlap fraction). If you’re 40% into the current minute, you count every request in the current window plus 60% of the previous one. Two integers per key, no boundary exploit, and close enough to exact for real-world limits.

Token bucket. A bucket holds up to capacity tokens and refills at a fixed rate per second. Each request takes a token; no token, no request. A client can burst up to capacity requests at once, then gets throttled to the refill rate. That matches how browsers and apps actually call APIs: a page load fires eight requests together, then nothing for a while. A flat per-second cap would reject half of that page load.

Leaky bucket. Same bucket, opposite goal. Incoming requests go into a queue and leave at a constant rate, so bursts come out as a steady stream. Token bucket decides how much burst a client is allowed. Leaky bucket decides how steady the outgoing rate is. Reach for it when whatever sits downstream, a third-party API with its own quota or a fixed-size worker pool, needs a constant feed.

How to build a rate limiter with Redis

The counter has to live somewhere every instance of your API can see. An in-process counter works until you run a second instance; then each one enforces the limit against half the traffic. Redis is the usual store (see How Redis Caching Works and How to Use It if you haven’t set one up), and the reason is its atomic commands more than its speed.

A fixed window counter in Python with INCR and EXPIRE:

1
2
3
4
5

def is_allowed(redis, key: str, limit: int, window_seconds: int) -> bool:
    count = redis.incr(f"rl:{key}")
    if count == 1:
        redis.expire(f"rl:{key}", window_seconds)
    return count <= limit

INCR is atomic on a single Redis instance, so two requests arriving in the same microsecond still get two distinct counts back. The bug most hand-rolled limiters ship is a GET followed by a SET. Two concurrent requests read the same count, both decide they’re under the limit, and both get through. One gap remains in the version above: if your process dies between INCR and EXPIRE, the key never expires and that client stays blocked. Moving both calls into a Lua script, like the token bucket below, closes it.

Token bucket needs three steps: compute the refill, take a token, write the state back. No single Redis command does all three, so you run them as a Lua script through EVAL, which Redis executes as one atomic unit:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

-- KEYS[1] = bucket key, ARGV[1] = capacity, ARGV[2] = refill rate/sec, ARGV[3] = now
local bucket = redis.call("HMGET", KEYS[1], "tokens", "ts")
local tokens = tonumber(bucket[1]) or tonumber(ARGV[1])
local last = tonumber(bucket[2]) or tonumber(ARGV[3])
local elapsed = tonumber(ARGV[3]) - last
tokens = math.min(tonumber(ARGV[1]), tokens + elapsed * tonumber(ARGV[2]))
local ttl = math.ceil(tonumber(ARGV[1]) / tonumber(ARGV[2]))

if tokens < 1 then
  redis.call("HSET", KEYS[1], "tokens", tokens, "ts", ARGV[3])
  redis.call("EXPIRE", KEYS[1], ttl)
  return 0
end

redis.call("HSET", KEYS[1], "tokens", tokens - 1, "ts", ARGV[3])
redis.call("EXPIRE", KEYS[1], ttl)
return 1

Call it with redis.eval(script, 1, f"bucket:{key}", capacity, refill_rate, time.time()). The script refills the bucket for the time that has passed, takes a token in the same pass, and sets a TTL long enough for a full refill — so a client that stops calling the API doesn’t leave its bucket sitting in Redis forever. No other request can read or change the bucket while it runs.

Which headers to send with a 429 response

A rejected request should tell the client when to come back.

Use 429 Too Many Requests, not 403. The client isn’t forbidden, it’s early.

Add Retry-After: 30, in seconds or as an HTTP date. Many HTTP client libraries and SDKs read it to schedule their retry, and the ones that don’t at least give a human something to go on.

Send X-RateLimit-Limit, X-RateLimit-Remaining and X-RateLimit-Reset on every response, not only the rejected ones. They tell the client the ceiling, the quota left, and when the window resets. This is a de facto convention (GitHub’s API uses it) rather than a standard, but it’s what client developers look for. A well-behaved client watches Remaining drop and slows down before it ever sees a 429.

A standard version exists in draft: the RateLimit and RateLimit-Policy fields in the IETF’s draft-ietf-httpapi-ratelimit-headers. It has been an Internet-Draft for several years and is not yet an RFC. Ship the X-RateLimit-* headers now, and add the standard fields when the draft is published as an RFC.

Common rate limiting mistakes

Limiting by IP alone. A corporate network, a mobile carrier’s NAT or a CDN edge can put thousands of real users behind one address. Key the limit on the API key or authenticated user ID wherever you have one, and fall back to IP only for anonymous traffic. JWT vs Session Authentication: How to Choose covers how that identity gets established.

An in-memory counter behind a load balancer. Each instance counts on its own, so a limit of 100 turns into 100 Ă— the number of instances. Once you scale horizontally, the counter belongs in Redis, not in a process-local dict.

One limit for the whole API. A search request and a bulk export don’t cost the same to serve. Set limits per route, or at least per cost class.

Zero burst tolerance. A webhook retry, a bulk import, a mobile app reconnecting after a tunnel: to a flat per-second cap these all look like abuse. Token bucket exists for exactly this traffic.

Which rate limiter should you build first?

For a typical read-heavy API, build a sliding window counter keyed on API key, stored in Redis, with X-RateLimit-* headers on every response. It avoids the fixed window’s boundary burst and the memory cost of a full request log. Add a token bucket only on the routes where clients legitimately send batches (webhook delivery, bulk uploads), and set its capacity from the size of the largest batch you expect, not from the steady-state rate.