{"schemaVersion":1,"id":"https://lucioduran.com/blog/rate-limiting-algorithms-token-bucket-sliding-window","url":"https://lucioduran.com/blog/rate-limiting-algorithms-token-bucket-sliding-window","title":"Rate Limiting Algorithms: From Token Buckets to Distributed Lua Scripts","description":"Token bucket, sliding window log, GCRA, distributed rate limiting with Redis, the algorithms behind every API gateway, with production-ready Lua script implementations.","language":"en","datePublished":"2025-10-21","dateModified":null,"contentFormat":"text/markdown","content":"Backend\n\n# Rate Limiting Algorithms: From Token Buckets to Distributed Lua Scripts\n\n2025-10-218 min read\n\nLucio Durán\nEngineering Manager & AI Solutions Architect\n\nAlso available in: [Español](<https://lucioduran.com/es/blog/rate-limiting-algorithms-token-bucket-sliding-window>)\n\n## Fixed window counter: the naive approach\n\nThe simplest rate limiter. Divide time into fixed windows (e.g., 1-minute intervals), count requests per window, reject when the count exceeds the limit.\n\n```python\nimport time\nimport redis\n\nr = redis.Redis()\n\ndef fixed_window(key: str, limit: int, window_seconds: int) -> bool:\n \"\"\"Returns True if request is allowed\"\"\"\n window = int(time.time() / window_seconds)\n redis_key = f\"ratelimit:{key}:{window}\"\n\n current = r.incr(redis_key)\n if current == 1:\n r.expire(redis_key, window_seconds + 1)\n\n return current <= limit\n```\n\nThe problem: boundary burst. At the edge of two windows, a client can make ` 2 * limit ` requests in a span of ` window_seconds `. If your limit is 100/minute, a client can fire 100 requests at 11:59:59 and another 100 at 12:00:00, 200 requests in 2 seconds while \"respecting\" the limit.\n\n## Sliding window log: exact but expensive\n\nKeep a sorted set of every request timestamp. For each new request, remove timestamps older than the window, count remaining entries, and decide.\n\n```lua\n-- sliding_window_log.lua\n-- KEYS[1] = rate limit key\n-- ARGV[1] = current timestamp (ms)\n-- ARGV[2] = window size (ms)\n-- ARGV[3] = max requests\n\nlocal key = KEYS[1]\nlocal now = tonumber(ARGV[1])\nlocal window = tonumber(ARGV[2])\nlocal limit = tonumber(ARGV[3])\n\n-- Remove entries outside the window\nredis.call('ZREMRANGEBYSCORE', key, 0, now - window)\n\n-- Count current entries\nlocal count = redis.call('ZCARD', key)\n\nif count < limit then\n -- Add this request\n redis.call('ZADD', key, now, now .. ':' .. math.random(1000000))\n redis.call('PEXPIRE', key, window)\n return {1, limit - count - 1} -- allowed, remaining\nelse\n return {0, 0} -- denied, remaining\nend\n```\n\nThis gives exact counts, no boundary burst problem. But it stores every request timestamp. At 1,000 req/s with a 60-second window, that's 60,000 entries per key. Memory usage becomes significant at scale.\n\n## Sliding window counter: the practical middle ground\n\nCombine two fixed windows with weighted counting. Estimate the current window's count by interpolating between the previous and current windows:\n\n```lua\n-- sliding_window_counter.lua\nlocal key = KEYS[1]\nlocal limit = tonumber(ARGV[1])\nlocal window = tonumber(ARGV[2])\nlocal now = tonumber(ARGV[3])\n\nlocal current_window = math.floor(now / window)\nlocal previous_window = current_window - 1\nlocal elapsed = (now % window) / window -- 0.0 to 1.0\n\nlocal current_key = key .. ':' .. current_window\nlocal previous_key = key .. ':' .. previous_window\n\nlocal current_count = tonumber(redis.call('GET', current_key) or '0')\nlocal previous_count = tonumber(redis.call('GET', previous_key) or '0')\n\n-- Weighted estimate\nlocal estimated = previous_count * (1 - elapsed) + current_count\n\nif estimated < limit then\n redis.call('INCR', current_key)\n redis.call('PEXPIRE', current_key, window * 2)\n return {1, math.floor(limit - estimated - 1)}\nelse\n return {0, 0}\nend\n```\n\nTwo keys per rate limit, O(1) operations, and the boundary burst problem is reduced (not eliminated, but reduced to acceptable levels for most use cases).\n\n## Token bucket: allowing controlled bursts\n\nThe token bucket is the classic algorithm. Tokens accumulate at a fixed rate up to a maximum (the bucket size). Each request consumes one token. If the bucket is empty, the request is denied.\n\n```lua\n-- token_bucket.lua\nlocal key = KEYS[1]\nlocal capacity = tonumber(ARGV[1]) -- max tokens\nlocal rate = tonumber(ARGV[2]) -- tokens per second\nlocal now = tonumber(ARGV[3]) -- current timestamp (ms)\nlocal requested = tonumber(ARGV[4]) -- tokens to consume (usually 1)\n\nlocal data = redis.call('HMGET', key, 'tokens', 'last_refill')\nlocal tokens = tonumber(data[1])\nlocal last_refill = tonumber(data[2])\n\nif tokens == nil then\n -- First request: initialize full bucket\n tokens = capacity\n last_refill = now\nend\n\n-- Calculate tokens to add since last refill\nlocal elapsed = (now - last_refill) / 1000.0\nlocal new_tokens = elapsed * rate\ntokens = math.min(capacity, tokens + new_tokens)\n\nlocal allowed = 0\nlocal remaining = tokens\n\nif tokens >= requested then\n tokens = tokens - requested\n allowed = 1\n remaining = tokens\nend\n\nredis.call('HMSET', key, 'tokens', tokens, 'last_refill', now)\nredis.call('PEXPIRE', key, math.ceil(capacity / rate * 1000) + 1000)\n\nreturn {allowed, math.floor(remaining)}\n```\n\nThe token bucket has a nice property: it allows bursts. If a client has been idle, tokens accumulate and they can make several rapid requests. This is desirable for many API patterns, a client that polls every 10 seconds doesn't need perfectly smooth limiting.\n\nBut bursts are sometimes exactly what you don't want. That's where GCRA comes in.\n\n## GCRA: the cell rate algorithm\n\nGCRA (Generic Cell Rate Algorithm) comes from ATM network traffic shaping. Instead of tracking tokens, it tracks when the next request should be allowed. Each request advances this \"theoretical arrival time\" (TAT) by a fixed increment.\n\n```lua\n-- gcra.lua\nlocal key = KEYS[1]\nlocal emission_interval = tonumber(ARGV[1]) -- ms between allowed requests\nlocal burst_offset = tonumber(ARGV[2]) -- max burst tolerance (ms)\nlocal now = tonumber(ARGV[3]) -- current timestamp (ms)\n\nlocal tat = tonumber(redis.call('GET', key)) -- theoretical arrival time\n\nif tat == nil then\n -- First request: allow and set TAT\n tat = now + emission_interval\n redis.call('SET', key, tat)\n redis.call('PEXPIRE', key, burst_offset + emission_interval)\n return {1, 0} -- allowed, retry_after=0\nend\n\nlocal allow_at = tat - burst_offset\n\nif now < allow_at then\n -- Too early: request denied\n local retry_after = allow_at - now\n return {0, retry_after}\nelse\n -- Allowed: advance TAT\n local new_tat = math.max(now, tat) + emission_interval\n redis.call('SET', key, new_tat)\n redis.call('PEXPIRE', key, burst_offset + emission_interval)\n return {1, 0}\nend\n```\n\nFor 100 requests per second, ` emission_interval = 10ms ` (one request every 10ms). The ` burst_offset ` allows some flexibility, setting it to 100ms lets clients send up to 10 requests at once before the spacing kicks in.\n\nGCRA has favorable properties: single key per client, O(1) operations, no boundary bursts, smooth rate enforcement. [Cloudflare](<https://www.cloudflare.com/>) uses this algorithm internally for their rate limiting product.\n\n## The distributed race condition\n\nA common and dangerous bug in distributed rate limiting involves a race condition. Here is the broken implementation:\n\n```python\n# BROKEN: race condition between check and decrement\ndef check_rate_limit(key, limit, window):\n count = redis.get(key) or 0\n if int(count) >= limit:\n return False\n # Between this GET and the INCR, other requests can pass!\n redis.incr(key)\n return True\n```\n\nWith 100 concurrent requests hitting this code simultaneously, all 100 read count=0, all 100 pass the check, then all 100 increment. The limit is meaningless.\n\nThe fix: atomic operations via Lua scripts. Every Lua script in Redis executes atomically, no other command can interleave. All the Lua implementations above are atomic by design. Load them once with ` SCRIPT LOAD `, then call with ` EVALSHA `:\n\n```python\nimport redis\nimport hashlib\n\nclass DistributedRateLimiter:\n def __init__(self, redis_client: redis.Redis):\n self.r = redis_client\n self._script_sha = None\n\n def _load_script(self):\n script = \"\"\"\n local key = KEYS[1]\n local capacity = tonumber(ARGV[1])\n local rate = tonumber(ARGV[2])\n local now = tonumber(ARGV[3])\n -- ... (token bucket implementation from above)\n \"\"\"\n self._script_sha = self.r.script_load(script)\n\n def allow(self, key: str, capacity: int = 100, rate: float = 10.0) -> tuple[bool, int]:\n if self._script_sha is None:\n self._load_script()\n\n now_ms = int(time.time() * 1000)\n try:\n result = self.r.evalsha(\n self._script_sha, 1, key,\n capacity, rate, now_ms, 1\n )\n except redis.exceptions.NoScriptError:\n self._load_script()\n result = self.r.evalsha(\n self._script_sha, 1, key,\n capacity, rate, now_ms, 1\n )\n\n allowed = bool(result[0])\n remaining = int(result[1])\n return allowed, remaining\n```\n\nThe ` NoScriptError ` handling is important, if Redis restarts, cached scripts are lost. The retry with ` _load_script() ` handles this transparently.\n\n## Response headers: be a good citizen\n\nAlways include rate limit information in your HTTP responses:\n\n```python\nfrom fastapi import FastAPI, Request, Response\nfrom fastapi.responses import JSONResponse\n\napp = FastAPI()\nlimiter = DistributedRateLimiter(redis_client)\n\n@app.middleware(\"http\")\nasync def rate_limit_middleware(request: Request, call_next):\n key = f\"api:{request.client.host}:{request.url.path}\"\n allowed, remaining = limiter.allow(key, capacity=100, rate=10.0)\n\n if not allowed:\n return JSONResponse(\n status_code=429,\n content={\"error\": \"Rate limit exceeded\"},\n headers={\n \"X-RateLimit-Limit\": \"100\",\n \"X-RateLimit-Remaining\": \"0\",\n \"X-RateLimit-Reset\": str(int(time.time()) + 10),\n \"Retry-After\": \"10\",\n }\n )\n\n response = await call_next(request)\n response.headers[\"X-RateLimit-Limit\"] = \"100\"\n response.headers[\"X-RateLimit-Remaining\"] = str(remaining)\n response.headers[\"X-RateLimit-Reset\"] = str(int(time.time()) + 10)\n return response\n```\n\nThe ` Retry-After ` header on 429 responses is particularly important. Well-behaved clients (SDKs, scripts) use it to back off automatically. Without it, they retry immediately and make the situation worse.\n\n## Multi-tier rate limiting\n\nIn production, a three-tier approach works well:\n\n1. **Edge (Cloudflare/nginx)**: IP-based, high limits (1,000/min), catches DDoS and scrapers\n2. **API Gateway**: API key-based, per-plan limits (free: 60/min, pro: 600/min)\n3. **Per-endpoint**: Specific limits for expensive operations (search: 10/min, export: 5/min)\n\nEach tier runs independently with its own Redis key namespace. A request must pass all three to proceed. This layering means you can be generous at the edge (catching only obvious abuse) and strict at the endpoint level (protecting expensive resources).\n\nAfter switching to atomic Lua scripts, the rate limiter becomes bulletproof. Twelve months of production use, zero incidents, zero race conditions. The implementation is less than 50 lines of Lua. Sometimes the simplest fix is the best one.\n\n## Frequently asked questions\n\n### Which rate limiting algorithm should I use?\n\nFor most API rate limiting, GCRA or sliding window counter. Token bucket if you want to allow controlled bursts. Fixed window only for simple, non-critical limits. Sliding window log for exact counting when memory isn't a concern.\n\n### How do I rate limit in a distributed system?\n\nUse Redis with atomic Lua scripts. Each rate limit check should be a single EVALSHA call that reads the current state, decides allow/deny, and updates state atomically. This avoids race conditions between distributed application instances.\n\n### Should I rate limit by IP or by API key?\n\nBoth, at different layers. IP-based limiting at the edge (Cloudflare, nginx) catches DDoS and unauthenticated abuse. API key limiting at the application layer enforces per-customer quotas. The two are complementary.\n\n## Tips\n\n1. **Tip:** Use GCRA (Generic Cell Rate Algorithm) over token bucket when you need smooth rate limiting. Token bucket allows bursts up to the bucket size; GCRA spaces requests evenly, which is what you actually want for API protection.\n2. **Tip:** Always include ` X-RateLimit-Limit `, ` X-RateLimit-Remaining `, and ` X-RateLimit-Reset ` headers in your responses. Clients that respect these headers reduce your load voluntarily, free capacity for everyone.\n3. **Tip:** In distributed setups, use a single Redis ` EVALSHA ` call for check-and-decrement. Two separate commands (GET then SET) create a race condition where concurrent requests both pass the check before either decrements.\n4. **Tip:** Set your Redis key TTL to ` window_size + 1 second `, not exactly ` window_size `. Clock skew between application servers and Redis can cause keys to expire one request too early.\n\n- rate-limiting\n- token-bucket\n- sliding-window\n- gcra\n- redis\n- lua\n- api-gateway\n- distributed-systems\n\n[Back to Blog](<https://lucioduran.com/blog>)","headings":[{"level":1,"text":"Rate Limiting Algorithms: From Token Buckets to Distributed Lua Scripts","url":"https://lucioduran.com/blog/rate-limiting-algorithms-token-bucket-sliding-window#title"},{"level":2,"text":"Fixed window counter: the naive approach","url":"https://lucioduran.com/blog/rate-limiting-algorithms-token-bucket-sliding-window#fixed-window-counter-the-naive-approach"},{"level":2,"text":"Sliding window log: exact but expensive","url":"https://lucioduran.com/blog/rate-limiting-algorithms-token-bucket-sliding-window#sliding-window-log-exact-but-expensive"},{"level":2,"text":"Sliding window counter: the practical middle ground","url":"https://lucioduran.com/blog/rate-limiting-algorithms-token-bucket-sliding-window#sliding-window-counter-the-practical-middle-ground"},{"level":2,"text":"Token bucket: allowing controlled bursts","url":"https://lucioduran.com/blog/rate-limiting-algorithms-token-bucket-sliding-window#token-bucket-allowing-controlled-bursts"},{"level":2,"text":"GCRA: the cell rate algorithm","url":"https://lucioduran.com/blog/rate-limiting-algorithms-token-bucket-sliding-window#gcra-the-cell-rate-algorithm"},{"level":2,"text":"The distributed race condition","url":"https://lucioduran.com/blog/rate-limiting-algorithms-token-bucket-sliding-window#the-distributed-race-condition"},{"level":2,"text":"Response headers: be a good citizen","url":"https://lucioduran.com/blog/rate-limiting-algorithms-token-bucket-sliding-window#response-headers-be-a-good-citizen"},{"level":2,"text":"Multi-tier rate limiting","url":"https://lucioduran.com/blog/rate-limiting-algorithms-token-bucket-sliding-window#multi-tier-rate-limiting"},{"level":2,"text":"Frequently asked questions","url":"https://lucioduran.com/blog/rate-limiting-algorithms-token-bucket-sliding-window#questions"},{"level":3,"text":"Which rate limiting algorithm should I use?","url":"https://lucioduran.com/blog/rate-limiting-algorithms-token-bucket-sliding-window#question-1"},{"level":3,"text":"How do I rate limit in a distributed system?","url":"https://lucioduran.com/blog/rate-limiting-algorithms-token-bucket-sliding-window#question-2"},{"level":3,"text":"Should I rate limit by IP or by API key?","url":"https://lucioduran.com/blog/rate-limiting-algorithms-token-bucket-sliding-window#question-3"},{"level":2,"text":"Tips","url":"https://lucioduran.com/blog/rate-limiting-algorithms-token-bucket-sliding-window#article-tips"}],"sections":[{"heading":"Rate Limiting Algorithms: From Token Buckets to Distributed Lua Scripts","level":1,"url":"https://lucioduran.com/blog/rate-limiting-algorithms-token-bucket-sliding-window","content":"Backend"},{"heading":"Rate Limiting Algorithms: From Token Buckets to Distributed Lua Scripts","level":1,"url":"https://lucioduran.com/blog/rate-limiting-algorithms-token-bucket-sliding-window#title","content":"2025-10-218 min read\n\nLucio Durán\nEngineering Manager & AI Solutions Architect\n\nAlso available in: [Español](<https://lucioduran.com/es/blog/rate-limiting-algorithms-token-bucket-sliding-window>)"},{"heading":"Fixed window counter: the naive approach","level":2,"url":"https://lucioduran.com/blog/rate-limiting-algorithms-token-bucket-sliding-window#fixed-window-counter-the-naive-approach","content":"The simplest rate limiter. Divide time into fixed windows (e.g., 1-minute intervals), count requests per window, reject when the count exceeds the limit.\n\n```python\nimport time\nimport redis\n\nr = redis.Redis()\n\ndef fixed_window(key: str, limit: int, window_seconds: int) -> bool:\n \"\"\"Returns True if request is allowed\"\"\"\n window = int(time.time() / window_seconds)\n redis_key = f\"ratelimit:{key}:{window}\"\n\n current = r.incr(redis_key)\n if current == 1:\n r.expire(redis_key, window_seconds + 1)\n\n return current <= limit\n```\n\nThe problem: boundary burst. At the edge of two windows, a client can make ` 2 * limit ` requests in a span of ` window_seconds `. If your limit is 100/minute, a client can fire 100 requests at 11:59:59 and another 100 at 12:00:00, 200 requests in 2 seconds while \"respecting\" the limit."},{"heading":"Sliding window log: exact but expensive","level":2,"url":"https://lucioduran.com/blog/rate-limiting-algorithms-token-bucket-sliding-window#sliding-window-log-exact-but-expensive","content":"Keep a sorted set of every request timestamp. For each new request, remove timestamps older than the window, count remaining entries, and decide.\n\n```lua\n-- sliding_window_log.lua\n-- KEYS[1] = rate limit key\n-- ARGV[1] = current timestamp (ms)\n-- ARGV[2] = window size (ms)\n-- ARGV[3] = max requests\n\nlocal key = KEYS[1]\nlocal now = tonumber(ARGV[1])\nlocal window = tonumber(ARGV[2])\nlocal limit = tonumber(ARGV[3])\n\n-- Remove entries outside the window\nredis.call('ZREMRANGEBYSCORE', key, 0, now - window)\n\n-- Count current entries\nlocal count = redis.call('ZCARD', key)\n\nif count < limit then\n -- Add this request\n redis.call('ZADD', key, now, now .. ':' .. math.random(1000000))\n redis.call('PEXPIRE', key, window)\n return {1, limit - count - 1} -- allowed, remaining\nelse\n return {0, 0} -- denied, remaining\nend\n```\n\nThis gives exact counts, no boundary burst problem. But it stores every request timestamp. At 1,000 req/s with a 60-second window, that's 60,000 entries per key. Memory usage becomes significant at scale."},{"heading":"Sliding window counter: the practical middle ground","level":2,"url":"https://lucioduran.com/blog/rate-limiting-algorithms-token-bucket-sliding-window#sliding-window-counter-the-practical-middle-ground","content":"Combine two fixed windows with weighted counting. Estimate the current window's count by interpolating between the previous and current windows:\n\n```lua\n-- sliding_window_counter.lua\nlocal key = KEYS[1]\nlocal limit = tonumber(ARGV[1])\nlocal window = tonumber(ARGV[2])\nlocal now = tonumber(ARGV[3])\n\nlocal current_window = math.floor(now / window)\nlocal previous_window = current_window - 1\nlocal elapsed = (now % window) / window -- 0.0 to 1.0\n\nlocal current_key = key .. ':' .. current_window\nlocal previous_key = key .. ':' .. previous_window\n\nlocal current_count = tonumber(redis.call('GET', current_key) or '0')\nlocal previous_count = tonumber(redis.call('GET', previous_key) or '0')\n\n-- Weighted estimate\nlocal estimated = previous_count * (1 - elapsed) + current_count\n\nif estimated < limit then\n redis.call('INCR', current_key)\n redis.call('PEXPIRE', current_key, window * 2)\n return {1, math.floor(limit - estimated - 1)}\nelse\n return {0, 0}\nend\n```\n\nTwo keys per rate limit, O(1) operations, and the boundary burst problem is reduced (not eliminated, but reduced to acceptable levels for most use cases)."},{"heading":"Token bucket: allowing controlled bursts","level":2,"url":"https://lucioduran.com/blog/rate-limiting-algorithms-token-bucket-sliding-window#token-bucket-allowing-controlled-bursts","content":"The token bucket is the classic algorithm. Tokens accumulate at a fixed rate up to a maximum (the bucket size). Each request consumes one token. If the bucket is empty, the request is denied.\n\n```lua\n-- token_bucket.lua\nlocal key = KEYS[1]\nlocal capacity = tonumber(ARGV[1]) -- max tokens\nlocal rate = tonumber(ARGV[2]) -- tokens per second\nlocal now = tonumber(ARGV[3]) -- current timestamp (ms)\nlocal requested = tonumber(ARGV[4]) -- tokens to consume (usually 1)\n\nlocal data = redis.call('HMGET', key, 'tokens', 'last_refill')\nlocal tokens = tonumber(data[1])\nlocal last_refill = tonumber(data[2])\n\nif tokens == nil then\n -- First request: initialize full bucket\n tokens = capacity\n last_refill = now\nend\n\n-- Calculate tokens to add since last refill\nlocal elapsed = (now - last_refill) / 1000.0\nlocal new_tokens = elapsed * rate\ntokens = math.min(capacity, tokens + new_tokens)\n\nlocal allowed = 0\nlocal remaining = tokens\n\nif tokens >= requested then\n tokens = tokens - requested\n allowed = 1\n remaining = tokens\nend\n\nredis.call('HMSET', key, 'tokens', tokens, 'last_refill', now)\nredis.call('PEXPIRE', key, math.ceil(capacity / rate * 1000) + 1000)\n\nreturn {allowed, math.floor(remaining)}\n```\n\nThe token bucket has a nice property: it allows bursts. If a client has been idle, tokens accumulate and they can make several rapid requests. This is desirable for many API patterns, a client that polls every 10 seconds doesn't need perfectly smooth limiting.\n\nBut bursts are sometimes exactly what you don't want. That's where GCRA comes in."},{"heading":"GCRA: the cell rate algorithm","level":2,"url":"https://lucioduran.com/blog/rate-limiting-algorithms-token-bucket-sliding-window#gcra-the-cell-rate-algorithm","content":"GCRA (Generic Cell Rate Algorithm) comes from ATM network traffic shaping. Instead of tracking tokens, it tracks when the next request should be allowed. Each request advances this \"theoretical arrival time\" (TAT) by a fixed increment.\n\n```lua\n-- gcra.lua\nlocal key = KEYS[1]\nlocal emission_interval = tonumber(ARGV[1]) -- ms between allowed requests\nlocal burst_offset = tonumber(ARGV[2]) -- max burst tolerance (ms)\nlocal now = tonumber(ARGV[3]) -- current timestamp (ms)\n\nlocal tat = tonumber(redis.call('GET', key)) -- theoretical arrival time\n\nif tat == nil then\n -- First request: allow and set TAT\n tat = now + emission_interval\n redis.call('SET', key, tat)\n redis.call('PEXPIRE', key, burst_offset + emission_interval)\n return {1, 0} -- allowed, retry_after=0\nend\n\nlocal allow_at = tat - burst_offset\n\nif now < allow_at then\n -- Too early: request denied\n local retry_after = allow_at - now\n return {0, retry_after}\nelse\n -- Allowed: advance TAT\n local new_tat = math.max(now, tat) + emission_interval\n redis.call('SET', key, new_tat)\n redis.call('PEXPIRE', key, burst_offset + emission_interval)\n return {1, 0}\nend\n```\n\nFor 100 requests per second, ` emission_interval = 10ms ` (one request every 10ms). The ` burst_offset ` allows some flexibility, setting it to 100ms lets clients send up to 10 requests at once before the spacing kicks in.\n\nGCRA has favorable properties: single key per client, O(1) operations, no boundary bursts, smooth rate enforcement. [Cloudflare](<https://www.cloudflare.com/>) uses this algorithm internally for their rate limiting product."},{"heading":"The distributed race condition","level":2,"url":"https://lucioduran.com/blog/rate-limiting-algorithms-token-bucket-sliding-window#the-distributed-race-condition","content":"A common and dangerous bug in distributed rate limiting involves a race condition. Here is the broken implementation:\n\n```python\n# BROKEN: race condition between check and decrement\ndef check_rate_limit(key, limit, window):\n count = redis.get(key) or 0\n if int(count) >= limit:\n return False\n # Between this GET and the INCR, other requests can pass!\n redis.incr(key)\n return True\n```\n\nWith 100 concurrent requests hitting this code simultaneously, all 100 read count=0, all 100 pass the check, then all 100 increment. The limit is meaningless.\n\nThe fix: atomic operations via Lua scripts. Every Lua script in Redis executes atomically, no other command can interleave. All the Lua implementations above are atomic by design. Load them once with ` SCRIPT LOAD `, then call with ` EVALSHA `:\n\n```python\nimport redis\nimport hashlib\n\nclass DistributedRateLimiter:\n def __init__(self, redis_client: redis.Redis):\n self.r = redis_client\n self._script_sha = None\n\n def _load_script(self):\n script = \"\"\"\n local key = KEYS[1]\n local capacity = tonumber(ARGV[1])\n local rate = tonumber(ARGV[2])\n local now = tonumber(ARGV[3])\n -- ... (token bucket implementation from above)\n \"\"\"\n self._script_sha = self.r.script_load(script)\n\n def allow(self, key: str, capacity: int = 100, rate: float = 10.0) -> tuple[bool, int]:\n if self._script_sha is None:\n self._load_script()\n\n now_ms = int(time.time() * 1000)\n try:\n result = self.r.evalsha(\n self._script_sha, 1, key,\n capacity, rate, now_ms, 1\n )\n except redis.exceptions.NoScriptError:\n self._load_script()\n result = self.r.evalsha(\n self._script_sha, 1, key,\n capacity, rate, now_ms, 1\n )\n\n allowed = bool(result[0])\n remaining = int(result[1])\n return allowed, remaining\n```\n\nThe ` NoScriptError ` handling is important, if Redis restarts, cached scripts are lost. The retry with ` _load_script() ` handles this transparently."},{"heading":"Response headers: be a good citizen","level":2,"url":"https://lucioduran.com/blog/rate-limiting-algorithms-token-bucket-sliding-window#response-headers-be-a-good-citizen","content":"Always include rate limit information in your HTTP responses:\n\n```python\nfrom fastapi import FastAPI, Request, Response\nfrom fastapi.responses import JSONResponse\n\napp = FastAPI()\nlimiter = DistributedRateLimiter(redis_client)\n\n@app.middleware(\"http\")\nasync def rate_limit_middleware(request: Request, call_next):\n key = f\"api:{request.client.host}:{request.url.path}\"\n allowed, remaining = limiter.allow(key, capacity=100, rate=10.0)\n\n if not allowed:\n return JSONResponse(\n status_code=429,\n content={\"error\": \"Rate limit exceeded\"},\n headers={\n \"X-RateLimit-Limit\": \"100\",\n \"X-RateLimit-Remaining\": \"0\",\n \"X-RateLimit-Reset\": str(int(time.time()) + 10),\n \"Retry-After\": \"10\",\n }\n )\n\n response = await call_next(request)\n response.headers[\"X-RateLimit-Limit\"] = \"100\"\n response.headers[\"X-RateLimit-Remaining\"] = str(remaining)\n response.headers[\"X-RateLimit-Reset\"] = str(int(time.time()) + 10)\n return response\n```\n\nThe ` Retry-After ` header on 429 responses is particularly important. Well-behaved clients (SDKs, scripts) use it to back off automatically. Without it, they retry immediately and make the situation worse."},{"heading":"Multi-tier rate limiting","level":2,"url":"https://lucioduran.com/blog/rate-limiting-algorithms-token-bucket-sliding-window#multi-tier-rate-limiting","content":"In production, a three-tier approach works well:\n\n1. **Edge (Cloudflare/nginx)**: IP-based, high limits (1,000/min), catches DDoS and scrapers\n2. **API Gateway**: API key-based, per-plan limits (free: 60/min, pro: 600/min)\n3. **Per-endpoint**: Specific limits for expensive operations (search: 10/min, export: 5/min)\n\nEach tier runs independently with its own Redis key namespace. A request must pass all three to proceed. This layering means you can be generous at the edge (catching only obvious abuse) and strict at the endpoint level (protecting expensive resources).\n\nAfter switching to atomic Lua scripts, the rate limiter becomes bulletproof. Twelve months of production use, zero incidents, zero race conditions. The implementation is less than 50 lines of Lua. Sometimes the simplest fix is the best one."},{"heading":"Frequently asked questions","level":2,"url":"https://lucioduran.com/blog/rate-limiting-algorithms-token-bucket-sliding-window#questions","content":""},{"heading":"Which rate limiting algorithm should I use?","level":3,"url":"https://lucioduran.com/blog/rate-limiting-algorithms-token-bucket-sliding-window#question-1","content":"For most API rate limiting, GCRA or sliding window counter. Token bucket if you want to allow controlled bursts. Fixed window only for simple, non-critical limits. Sliding window log for exact counting when memory isn't a concern."},{"heading":"How do I rate limit in a distributed system?","level":3,"url":"https://lucioduran.com/blog/rate-limiting-algorithms-token-bucket-sliding-window#question-2","content":"Use Redis with atomic Lua scripts. Each rate limit check should be a single EVALSHA call that reads the current state, decides allow/deny, and updates state atomically. This avoids race conditions between distributed application instances."},{"heading":"Should I rate limit by IP or by API key?","level":3,"url":"https://lucioduran.com/blog/rate-limiting-algorithms-token-bucket-sliding-window#question-3","content":"Both, at different layers. IP-based limiting at the edge (Cloudflare, nginx) catches DDoS and unauthenticated abuse. API key limiting at the application layer enforces per-customer quotas. The two are complementary."},{"heading":"Tips","level":2,"url":"https://lucioduran.com/blog/rate-limiting-algorithms-token-bucket-sliding-window#article-tips","content":"1. **Tip:** Use GCRA (Generic Cell Rate Algorithm) over token bucket when you need smooth rate limiting. Token bucket allows bursts up to the bucket size; GCRA spaces requests evenly, which is what you actually want for API protection.\n2. **Tip:** Always include ` X-RateLimit-Limit `, ` X-RateLimit-Remaining `, and ` X-RateLimit-Reset ` headers in your responses. Clients that respect these headers reduce your load voluntarily, free capacity for everyone.\n3. **Tip:** In distributed setups, use a single Redis ` EVALSHA ` call for check-and-decrement. Two separate commands (GET then SET) create a race condition where concurrent requests both pass the check before either decrements.\n4. **Tip:** Set your Redis key TTL to ` window_size + 1 second `, not exactly ` window_size `. Clock skew between application servers and Redis can cause keys to expire one request too early.\n\n- rate-limiting\n- token-bucket\n- sliding-window\n- gcra\n- redis\n- lua\n- api-gateway\n- distributed-systems\n\n[Back to Blog](<https://lucioduran.com/blog>)"}],"links":[{"text":"Español","url":"https://lucioduran.com/es/blog/rate-limiting-algorithms-token-bucket-sliding-window"},{"text":"Cloudflare","url":"https://www.cloudflare.com/"},{"text":"Back to Blog","url":"https://lucioduran.com/blog"}],"structuredData":[{"@context":"https://schema.org","@graph":[{"@type":"BlogPosting","@id":"https://lucioduran.com/blog/rate-limiting-algorithms-token-bucket-sliding-window#article","mainEntityOfPage":{"@type":"WebPage","@id":"https://lucioduran.com/blog/rate-limiting-algorithms-token-bucket-sliding-window"},"headline":"Rate Limiting Algorithms: From Token Buckets to Distributed Lua Scripts","description":"Token bucket, sliding window log, GCRA, distributed rate limiting with Redis, the algorithms behind every API gateway, with production-ready Lua script implementations.","datePublished":"2025-10-21","author":{"@type":"Person","@id":"https://lucioduran.com/#person","name":"Lucio Durán","url":"https://lucioduran.com"},"publisher":{"@type":"Person","@id":"https://lucioduran.com/#person","name":"Lucio Durán","url":"https://lucioduran.com"},"image":"https://statics.lucioduran.com/images/lucioduran-social-card.jpg","url":"https://lucioduran.com/blog/rate-limiting-algorithms-token-bucket-sliding-window","inLanguage":"en","isPartOf":{"@id":"https://lucioduran.com/blog#blog"},"articleSection":"Backend","keywords":"rate-limiting, token-bucket, sliding-window, gcra, redis, lua, api-gateway, distributed-systems","wordCount":1394,"timeRequired":"PT8M","workTranslation":[{"@type":"BlogPosting","inLanguage":"es","url":"https://lucioduran.com/es/blog/rate-limiting-algorithms-token-bucket-sliding-window"}]},{"@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://lucioduran.com/"},{"@type":"ListItem","position":2,"name":"Blog","item":"https://lucioduran.com/blog"},{"@type":"ListItem","position":3,"name":"Rate Limiting Algorithms: From Token Buckets to Distributed Lua Scripts","item":"https://lucioduran.com/blog/rate-limiting-algorithms-token-bucket-sliding-window"}]},{"@type":"FAQPage","mainEntity":[{"@type":"Question","name":"Which rate limiting algorithm should I use?","acceptedAnswer":{"@type":"Answer","text":"For most API rate limiting, GCRA or sliding window counter. Token bucket if you want to allow controlled bursts. Fixed window only for simple, non-critical limits. Sliding window log for exact counting when memory isn't a concern."}},{"@type":"Question","name":"How do I rate limit in a distributed system?","acceptedAnswer":{"@type":"Answer","text":"Use Redis with atomic Lua scripts. Each rate limit check should be a single EVALSHA call that reads the current state, decides allow/deny, and updates state atomically. This avoids race conditions between distributed application instances."}},{"@type":"Question","name":"Should I rate limit by IP or by API key?","acceptedAnswer":{"@type":"Answer","text":"Both, at different layers. IP-based limiting at the edge (Cloudflare, nginx) catches DDoS and unauthenticated abuse. API key limiting at the application layer enforces per-customer quotas. The two are complementary."}}]}]}],"contentHash":"bc1d5993b03aad3507d5ab49dd1cb99d0d557787be921ca727a6e3e772def425","provenance":{"method":"Deterministic serialization of the server-rendered main element","source":"https://lucioduran.com/blog/rate-limiting-algorithms-token-bucket-sliding-window","hashAlgorithm":"sha256","hashScope":"JSON document fields before contentHash and provenance","sourceHTMLBytes":64129,"contentBytes":12408}}