{"schemaVersion":1,"id":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing","url":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing","title":"Load Balancing Algorithms: Maglev Hashing, Consistent Hashing with Bounded Loads, and P2C","description":"An analysis of modern load balancing algorithms, Google's Maglev hashing, consistent hashing with bounded loads, power of two choices, and production HAProxy/Envoy configurations.","language":"en","datePublished":"2026-01-30","dateModified":null,"contentFormat":"text/markdown","content":"Infrastructure\n\n# Load Balancing Algorithms: Maglev Hashing, Consistent Hashing with Bounded Loads, and P2C\n\n2026-01-309 min read\n\nLucio Durán\nEngineering Manager & AI Solutions Architect\n\nAlso available in: [Español](<https://lucioduran.com/es/blog/load-balancing-algorithms-maglev-consistent-hashing>)\n\n## Fundamentals: Why Round-Robin Fails\n\nBefore examining Maglev and P2C, it is useful to establish why round-robin fails in real-world scenarios.\n\nRound-robin assumes all requests cost the same and all backends have the same capacity. Neither is true in practice. A request that triggers a cache miss takes 10x longer than a cache hit. A backend running on a ` c5.xlarge ` handles twice the load of a ` t3.medium `. Round-robin ignores both factors.\n\n```python\n# simulation.py - Why round-robin fails with variable latency\nimport random\nimport statistics\nfrom dataclasses import dataclass, field\n\n@dataclass\nclass Backend:\n name: str\n capacity: int # max concurrent requests\n active: int = 0\n total_served: int = 0\n latencies: list = field(default_factory=list)\n\ndef simulate_round_robin(backends: list[Backend], requests: int):\n idx = 0\n for _ in range(requests):\n backend = backends[idx % len(backends)]\n latency = random.expovariate(1/50) if random.random() < 0.1 else random.expovariate(1/2)\n backend.active += 1\n backend.total_served += 1\n backend.latencies.append(latency)\n if backend.active > backend.capacity:\n backend.latencies[-1] *= 3 # Overloaded penalty\n backend.active = max(0, backend.active - 1)\n idx += 1\n\ndef simulate_least_conn(backends: list[Backend], requests: int):\n for _ in range(requests):\n backend = min(backends, key=lambda b: b.active)\n latency = random.expovariate(1/50) if random.random() < 0.1 else random.expovariate(1/2)\n backend.active += 1\n backend.total_served += 1\n backend.latencies.append(latency)\n if backend.active > backend.capacity:\n backend.latencies[-1] *= 3\n backend.active = max(0, backend.active - 1)\n\n# Run simulation\nrr_backends = [Backend(f\"rr-{i}\", capacity=50) for i in range(5)]\nlc_backends = [Backend(f\"lc-{i}\", capacity=50) for i in range(5)]\n\nsimulate_round_robin(rr_backends, 10000)\nsimulate_least_conn(lc_backends, 10000)\n\nfor label, backends in [(\"Round Robin\", rr_backends), (\"Least Conn\", lc_backends)]:\n p99s = [statistics.quantiles(b.latencies, n=100)[98] for b in backends]\n print(f\"{label}: p99 range = {min(p99s):.1f}ms - {max(p99s):.1f}ms\")\n# Typical output:\n# Round Robin: p99 range = 142.3ms - 148.7ms (all backends suffer equally)\n# Least Conn: p99 range = 4.8ms - 7.2ms (slow requests concentrated)\n```\n\n## Consistent Hashing: The Cache Locality Problem\n\nConsistent hashing maps both keys and servers onto a ring. A key is served by the next server clockwise on the ring. When a server is added or removed, only the keys that were assigned to it get remapped, the rest stay put.\n\nThe classic implementation uses virtual nodes to improve distribution:\n\n```go\n// consistent_hash.go\npackage lb\n\nimport (\n \"hash/crc32\"\n \"sort\"\n \"strconv\"\n \"sync\"\n)\n\ntype ConsistentHash struct {\n mu sync.RWMutex\n ring []uint32\n nodes map[uint32]string\n vnodeCount int\n}\n\nfunc NewConsistentHash(vnodes int) *ConsistentHash {\n return &ConsistentHash{\n nodes: make(map[uint32]string),\n vnodeCount: vnodes,\n }\n}\n\nfunc (ch *ConsistentHash) hash(key string) uint32 {\n return crc32.ChecksumIEEE([]byte(key))\n}\n\nfunc (ch *ConsistentHash) AddNode(node string) {\n ch.mu.Lock()\n defer ch.mu.Unlock()\n\n for i := 0; i < ch.vnodeCount; i++ {\n vkey := node + \"#\" + strconv.Itoa(i)\n h := ch.hash(vkey)\n ch.ring = append(ch.ring, h)\n ch.nodes[h] = node\n }\n sort.Slice(ch.ring, func(i, j int) bool { return ch.ring[i] < ch.ring[j] })\n}\n\nfunc (ch *ConsistentHash) GetNode(key string) string {\n ch.mu.RLock()\n defer ch.mu.RUnlock()\n\n if len(ch.ring) == 0 {\n return \"\"\n }\n\n h := ch.hash(key)\n idx := sort.Search(len(ch.ring), func(i int) bool { return ch.ring[i] >= h })\n if idx == len(ch.ring) {\n idx = 0\n }\n return ch.nodes[ch.ring[idx]]\n}\n```\n\nThe problem: hot keys. If 30% of your traffic hits a single cache key, consistent hashing sends 30% of traffic to one backend. That backend melts while the others idle.\n\n### Consistent Hashing with Bounded Loads\n\nGoogle's 2017 paper introduced a fix: cap the maximum load any backend can receive. When a backend reaches its bound, the request spills to the next node on the ring.\n\n```go\n// bounded_consistent_hash.go\nfunc (ch *BoundedConsistentHash) GetNode(key string, loads map[string]int, avgLoad float64, epsilon float64) string {\n ch.mu.RLock()\n defer ch.mu.RUnlock()\n\n maxLoad := int(avgLoad * (1 + epsilon))\n\n h := ch.hash(key)\n idx := sort.Search(len(ch.ring), func(i int) bool { return ch.ring[i] >= h })\n\n for i := 0; i < len(ch.ring); i++ {\n nodeIdx := (idx + i) % len(ch.ring)\n node := ch.nodes[ch.ring[nodeIdx]]\n\n if loads[node] < maxLoad {\n return node\n }\n // This node is at capacity, try the next one on the ring\n }\n\n // All nodes at capacity (shouldn't happen with proper epsilon)\n return ch.nodes[ch.ring[idx]]\n}\n```\n\nWith ` epsilon = 0.25 `, no backend gets more than 125% of the average load. You lose some consistency (hot-key requests might land on different backends during spikes), but you prevent the worst-case scenario.\n\n## Maglev Hashing: Google's Production Algorithm\n\nMaglev is Google's hashing algorithm used in their network load balancer. It generates a lookup table where each entry maps to a backend. The table construction ensures minimal disruption when backends change.\n\n```python\n# maglev.py - Simplified Maglev lookup table generation\ndef maglev_table(backends: list[str], table_size: int = 65537) -> list[int]:\n \"\"\"\n Generate a Maglev lookup table.\n table_size should be a prime number > len(backends).\n \"\"\"\n n = len(backends)\n\n # Generate permutation for each backend\n # offset and skip determine each backend's preferred positions\n permutations = []\n for backend in backends:\n h1 = hash(backend + \"_offset\") % table_size\n h2 = (hash(backend + \"_skip\") % (table_size - 1)) + 1\n perm = [(h1 + j * h2) % table_size for j in range(table_size)]\n permutations.append(perm)\n\n # Fill the table using the permutation preferences\n table = [-1] * table_size\n next_idx = [0] * n # Track each backend's next preferred position\n filled = 0\n\n while filled < table_size:\n for i in range(n):\n # Find backend i's next unfilled preferred position\n while table[permutations[i][next_idx[i]]] != -1:\n next_idx[i] += 1\n # Claim that position\n table[permutations[i][next_idx[i]]] = i\n next_idx[i] += 1\n filled += 1\n if filled == table_size:\n break\n\n return table\n\n# Usage\nbackends = [\"web-1\", \"web-2\", \"web-3\", \"web-4\", \"web-5\"]\ntable = maglev_table(backends)\n\n# Lookup: hash the request key, index into the table\ndef get_backend(key: str, table: list[int], backends: list[str]) -> str:\n idx = hash(key) % len(table)\n return backends[table[idx]]\n\n# When web-3 is removed, regenerate the table\n# Only ~1/n of entries change (compared to modular hashing where everything shifts)\nremaining = [\"web-1\", \"web-2\", \"web-4\", \"web-5\"]\nnew_table = maglev_table(remaining)\n```\n\nMaglev's advantage over ring-based consistent hashing: nearly perfect uniformity. With virtual nodes, ring-based hashing still has variance. Maglev fills a lookup table round-robin style per backend, so the distribution is almost exactly even.\n\n## Power of Two Choices (P2C)\n\nP2C is elegantly simple: instead of picking one random backend, pick two random backends and send the request to whichever has fewer active connections.\n\nThe theoretical result is remarkable: random selection gives O(log n / log log n) max load, but P2C gives O(log log n) max load. That extra random choice exponentially reduces the worst case.\n\n```python\n# p2c.py - Power of Two Choices\nimport random\n\nclass P2CBalancer:\n def __init__(self, backends: list[str]):\n self.backends = backends\n self.active_connections = {b: 0 for b in backends}\n\n def pick(self) -> str:\n \"\"\"Pick the less-loaded of two random backends.\"\"\"\n if len(self.backends) < 2:\n return self.backends[0]\n\n a, b = random.sample(self.backends, 2)\n\n if self.active_connections[a] <= self.active_connections[b]:\n chosen = a\n else:\n chosen = b\n\n self.active_connections[chosen] += 1\n return chosen\n\n def release(self, backend: str):\n self.active_connections[backend] = max(0, self.active_connections[backend] - 1)\n```\n\n## Real Production Configurations\n\n### [HAProxy](<https://www.haproxy.org/>)\n\n```haproxy\n# haproxy.cfg - Production load balancing configuration\n\nglobal\n maxconn 50000\n log stdout local0\n\ndefaults\n mode http\n timeout connect 5s\n timeout client 30s\n timeout server 30s\n option httplog\n option forwardfor\n\n# Backend with least connections - for API with variable latency\nbackend api_servers\n balance leastconn\n option httpchk GET /health\n http-check expect status 200\n\n # Slow start: new backends ramp up over 30 seconds\n default-server inter 3s fall 3 rise 2 slowstart 30s\n\n server api-1 10.0.1.10:8080 check weight 100\n server api-2 10.0.1.11:8080 check weight 100\n server api-3 10.0.1.12:8080 check weight 100\n server api-4 10.0.1.13:8080 check weight 50 # Smaller instance\n\n# Backend with consistent hashing + bounded loads - for cache affinity\nbackend cache_servers\n balance uri\n hash-type consistent sdbm\n hash-balance-factor 150 # No backend gets >150% of average\n\n option httpchk GET /health\n http-check expect status 200\n\n server cache-1 10.0.2.10:8080 check\n server cache-2 10.0.2.11:8080 check\n server cache-3 10.0.2.12:8080 check\n server cache-4 10.0.2.13:8080 check\n\n# Backend with P2C - for general workloads\nbackend web_servers\n balance random(2) # Power of two choices\n\n option httpchk GET /health\n http-check expect status 200\n\n server web-1 10.0.3.10:8080 check\n server web-2 10.0.3.11:8080 check\n server web-3 10.0.3.12:8080 check\n\nfrontend http_front\n bind *:80\n bind *:443 ssl crt /etc/ssl/cert.pem\n\n acl is_api path_beg /api/\n acl is_cached path_beg /products/ /categories/\n\n use_backend api_servers if is_api\n use_backend cache_servers if is_cached\n default_backend web_servers\n```\n\n### [Envoy](<https://www.envoyproxy.io/>)\n\n```yaml\n# envoy.yaml - Envoy proxy with multiple LB policies\nstatic_resources:\n clusters:\n - name: api_cluster\n type: STRICT_DNS\n lb_policy: LEAST_REQUEST\n least_request_lb_config:\n choice_count: 3 # P3C - pick best of 3\n health_checks:\n - timeout: 2s\n interval: 5s\n unhealthy_threshold: 3\n healthy_threshold: 2\n http_health_check:\n path: /health\n load_assignment:\n cluster_name: api_cluster\n endpoints:\n - lb_endpoints:\n - endpoint:\n address:\n socket_address:\n address: api-1.internal\n port_value: 8080\n - endpoint:\n address:\n socket_address:\n address: api-2.internal\n port_value: 8080\n\n - name: cache_cluster\n type: STRICT_DNS\n lb_policy: RING_HASH\n ring_hash_lb_config:\n minimum_ring_size: 1024\n maximum_ring_size: 8388608\n hash_function: XX_HASH\n health_checks:\n - timeout: 2s\n interval: 5s\n http_health_check:\n path: /health\n load_assignment:\n cluster_name: cache_cluster\n endpoints:\n - lb_endpoints:\n - endpoint:\n address:\n socket_address:\n address: cache-1.internal\n port_value: 8080\n - endpoint:\n address:\n socket_address:\n address: cache-2.internal\n port_value: 8080\n\n - name: websocket_cluster\n type: STRICT_DNS\n lb_policy: LEAST_REQUEST\n common_lb_config:\n healthy_panic_threshold:\n value: 50 # If <50% backends healthy, use all\n load_assignment:\n cluster_name: websocket_cluster\n endpoints:\n - lb_endpoints:\n - endpoint:\n address:\n socket_address:\n address: ws-1.internal\n port_value: 8080\n load_balancing_weight: 100\n - endpoint:\n address:\n socket_address:\n address: ws-2.internal\n port_value: 8080\n load_balancing_weight: 100\n```\n\nThe ` choice_count: 3 ` in Envoy's ` LEAST_REQUEST ` config gives you P3C, pick the best of three random backends. Research shows P3C gives marginal improvement over P2C, but it's free in Envoy so why not.\n\nThe ` healthy_panic_threshold ` for WebSocket backends is critical. Without it, if two of four backends fail health checks, Envoy only sends traffic to the two healthy ones, which might overload them. With the panic threshold at 50%, if more than half the backends fail, Envoy sends traffic to all of them (including \"unhealthy\" ones) on the theory that partial service is better than cascading failure.\n\n## Practical Recommendations\n\nAfter running these algorithms in production across different workloads, the following defaults emerge:\n\n- **Stateless HTTP APIs**: ` leastconn ` (HAProxy) or ` LEAST_REQUEST ` with ` choice_count: 2 ` (Envoy). Simple, effective, handles variable latency gracefully.\n- **Cache layers**: Consistent hashing with bounded loads. HAProxy's ` hash-balance-factor ` or a custom implementation. Cache hit rates matter enough to justify the complexity.\n- **WebSocket/gRPC streams**: Least connections, always. Long-lived connections need to balance by connection count, not by request count.\n- **General web traffic**: P2C. Almost as good as least connections, but without the overhead of tracking connection counts across all proxies in a distributed setup.\n\nP2C is perhaps the most impactful algorithm in this comparison. The realization that one extra random choice can exponentially reduce worst-case load is a powerful insight. Using ` random(2) ` in HAProxy for any backend without a strong reason to use something else is a pragmatic default.\n\n## Frequently asked questions\n\n### What load balancing algorithm should I use by default?\n\nRound-robin for stateless HTTP services. Least connections for services with variable request duration. Consistent hashing only when you need session affinity or cache locality. P2C (power of two choices) when you want simplicity with near-optimal load distribution.\n\n### What is Maglev hashing and when should I use it?\n\nMaglev is Google's consistent hashing algorithm designed for their network load balancer. It provides minimal disruption when backends change and extremely uniform distribution. Use it when you're building a custom L4 load balancer or when you need better distribution than ring-based consistent hashing.\n\n### How does consistent hashing with bounded loads work?\n\nIt's consistent hashing with a cap: no backend can receive more than (1 + epsilon) times the average load. When a backend hits its bound, the request spills over to the next backend on the ring. This prevents hot spots while maintaining most of the consistency benefits.\n\n## Tips\n\n1. **Tip:** Use ` leastconn ` for long-lived connections (WebSocket, gRPC streams) and ` roundrobin ` for short HTTP requests. Mixing them up is the most common load balancing mistake.\n2. **Tip:** Envoy's ` RING_HASH ` with ` minimum_ring_size: 1024 ` gives you consistent hashing that survives backend additions without reshuffling all connections.\n3. **Tip:** Power of Two Choices (P2C) outperforms pure random selection dramatically. HAProxy's ` random(2) ` gives you P2C with zero configuration complexity.\n4. **Tip:** Always set ` hash-balance-factor ` in HAProxy when using consistent hashing. Without it, a hot key sends 100% of traffic to one backend. A factor of 150 caps any backend at 1.5x average load.\n\n- load-balancing\n- maglev\n- consistent-hashing\n- p2c\n- haproxy\n- envoy\n- infrastructure\n\n[Back to Blog](<https://lucioduran.com/blog>)","headings":[{"level":1,"text":"Load Balancing Algorithms: Maglev Hashing, Consistent Hashing with Bounded Loads, and P2C","url":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing#title"},{"level":2,"text":"Fundamentals: Why Round-Robin Fails","url":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing#fundamentals-why-round-robin-fails"},{"level":2,"text":"Consistent Hashing: The Cache Locality Problem","url":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing#consistent-hashing-the-cache-locality-problem"},{"level":3,"text":"Consistent Hashing with Bounded Loads","url":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing#consistent-hashing-with-bounded-loads"},{"level":2,"text":"Maglev Hashing: Google's Production Algorithm","url":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing#maglev-hashing-googles-production-algorithm"},{"level":2,"text":"Power of Two Choices (P2C)","url":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing#power-of-two-choices-p2c"},{"level":2,"text":"Real Production Configurations","url":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing#real-production-configurations"},{"level":3,"text":"HAProxy","url":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing#haproxy"},{"level":3,"text":"Envoy","url":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing#envoy"},{"level":2,"text":"Practical Recommendations","url":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing#practical-recommendations"},{"level":2,"text":"Frequently asked questions","url":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing#questions"},{"level":3,"text":"What load balancing algorithm should I use by default?","url":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing#question-1"},{"level":3,"text":"What is Maglev hashing and when should I use it?","url":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing#question-2"},{"level":3,"text":"How does consistent hashing with bounded loads work?","url":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing#question-3"},{"level":2,"text":"Tips","url":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing#article-tips"}],"sections":[{"heading":"Load Balancing Algorithms: Maglev Hashing, Consistent Hashing with Bounded Loads, and P2C","level":1,"url":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing","content":"Infrastructure"},{"heading":"Load Balancing Algorithms: Maglev Hashing, Consistent Hashing with Bounded Loads, and P2C","level":1,"url":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing#title","content":"2026-01-309 min read\n\nLucio Durán\nEngineering Manager & AI Solutions Architect\n\nAlso available in: [Español](<https://lucioduran.com/es/blog/load-balancing-algorithms-maglev-consistent-hashing>)"},{"heading":"Fundamentals: Why Round-Robin Fails","level":2,"url":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing#fundamentals-why-round-robin-fails","content":"Before examining Maglev and P2C, it is useful to establish why round-robin fails in real-world scenarios.\n\nRound-robin assumes all requests cost the same and all backends have the same capacity. Neither is true in practice. A request that triggers a cache miss takes 10x longer than a cache hit. A backend running on a ` c5.xlarge ` handles twice the load of a ` t3.medium `. Round-robin ignores both factors.\n\n```python\n# simulation.py - Why round-robin fails with variable latency\nimport random\nimport statistics\nfrom dataclasses import dataclass, field\n\n@dataclass\nclass Backend:\n name: str\n capacity: int # max concurrent requests\n active: int = 0\n total_served: int = 0\n latencies: list = field(default_factory=list)\n\ndef simulate_round_robin(backends: list[Backend], requests: int):\n idx = 0\n for _ in range(requests):\n backend = backends[idx % len(backends)]\n latency = random.expovariate(1/50) if random.random() < 0.1 else random.expovariate(1/2)\n backend.active += 1\n backend.total_served += 1\n backend.latencies.append(latency)\n if backend.active > backend.capacity:\n backend.latencies[-1] *= 3 # Overloaded penalty\n backend.active = max(0, backend.active - 1)\n idx += 1\n\ndef simulate_least_conn(backends: list[Backend], requests: int):\n for _ in range(requests):\n backend = min(backends, key=lambda b: b.active)\n latency = random.expovariate(1/50) if random.random() < 0.1 else random.expovariate(1/2)\n backend.active += 1\n backend.total_served += 1\n backend.latencies.append(latency)\n if backend.active > backend.capacity:\n backend.latencies[-1] *= 3\n backend.active = max(0, backend.active - 1)\n\n# Run simulation\nrr_backends = [Backend(f\"rr-{i}\", capacity=50) for i in range(5)]\nlc_backends = [Backend(f\"lc-{i}\", capacity=50) for i in range(5)]\n\nsimulate_round_robin(rr_backends, 10000)\nsimulate_least_conn(lc_backends, 10000)\n\nfor label, backends in [(\"Round Robin\", rr_backends), (\"Least Conn\", lc_backends)]:\n p99s = [statistics.quantiles(b.latencies, n=100)[98] for b in backends]\n print(f\"{label}: p99 range = {min(p99s):.1f}ms - {max(p99s):.1f}ms\")\n# Typical output:\n# Round Robin: p99 range = 142.3ms - 148.7ms (all backends suffer equally)\n# Least Conn: p99 range = 4.8ms - 7.2ms (slow requests concentrated)\n```"},{"heading":"Consistent Hashing: The Cache Locality Problem","level":2,"url":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing#consistent-hashing-the-cache-locality-problem","content":"Consistent hashing maps both keys and servers onto a ring. A key is served by the next server clockwise on the ring. When a server is added or removed, only the keys that were assigned to it get remapped, the rest stay put.\n\nThe classic implementation uses virtual nodes to improve distribution:\n\n```go\n// consistent_hash.go\npackage lb\n\nimport (\n \"hash/crc32\"\n \"sort\"\n \"strconv\"\n \"sync\"\n)\n\ntype ConsistentHash struct {\n mu sync.RWMutex\n ring []uint32\n nodes map[uint32]string\n vnodeCount int\n}\n\nfunc NewConsistentHash(vnodes int) *ConsistentHash {\n return &ConsistentHash{\n nodes: make(map[uint32]string),\n vnodeCount: vnodes,\n }\n}\n\nfunc (ch *ConsistentHash) hash(key string) uint32 {\n return crc32.ChecksumIEEE([]byte(key))\n}\n\nfunc (ch *ConsistentHash) AddNode(node string) {\n ch.mu.Lock()\n defer ch.mu.Unlock()\n\n for i := 0; i < ch.vnodeCount; i++ {\n vkey := node + \"#\" + strconv.Itoa(i)\n h := ch.hash(vkey)\n ch.ring = append(ch.ring, h)\n ch.nodes[h] = node\n }\n sort.Slice(ch.ring, func(i, j int) bool { return ch.ring[i] < ch.ring[j] })\n}\n\nfunc (ch *ConsistentHash) GetNode(key string) string {\n ch.mu.RLock()\n defer ch.mu.RUnlock()\n\n if len(ch.ring) == 0 {\n return \"\"\n }\n\n h := ch.hash(key)\n idx := sort.Search(len(ch.ring), func(i int) bool { return ch.ring[i] >= h })\n if idx == len(ch.ring) {\n idx = 0\n }\n return ch.nodes[ch.ring[idx]]\n}\n```\n\nThe problem: hot keys. If 30% of your traffic hits a single cache key, consistent hashing sends 30% of traffic to one backend. That backend melts while the others idle."},{"heading":"Consistent Hashing with Bounded Loads","level":3,"url":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing#consistent-hashing-with-bounded-loads","content":"Google's 2017 paper introduced a fix: cap the maximum load any backend can receive. When a backend reaches its bound, the request spills to the next node on the ring.\n\n```go\n// bounded_consistent_hash.go\nfunc (ch *BoundedConsistentHash) GetNode(key string, loads map[string]int, avgLoad float64, epsilon float64) string {\n ch.mu.RLock()\n defer ch.mu.RUnlock()\n\n maxLoad := int(avgLoad * (1 + epsilon))\n\n h := ch.hash(key)\n idx := sort.Search(len(ch.ring), func(i int) bool { return ch.ring[i] >= h })\n\n for i := 0; i < len(ch.ring); i++ {\n nodeIdx := (idx + i) % len(ch.ring)\n node := ch.nodes[ch.ring[nodeIdx]]\n\n if loads[node] < maxLoad {\n return node\n }\n // This node is at capacity, try the next one on the ring\n }\n\n // All nodes at capacity (shouldn't happen with proper epsilon)\n return ch.nodes[ch.ring[idx]]\n}\n```\n\nWith ` epsilon = 0.25 `, no backend gets more than 125% of the average load. You lose some consistency (hot-key requests might land on different backends during spikes), but you prevent the worst-case scenario."},{"heading":"Maglev Hashing: Google's Production Algorithm","level":2,"url":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing#maglev-hashing-googles-production-algorithm","content":"Maglev is Google's hashing algorithm used in their network load balancer. It generates a lookup table where each entry maps to a backend. The table construction ensures minimal disruption when backends change.\n\n```python\n# maglev.py - Simplified Maglev lookup table generation\ndef maglev_table(backends: list[str], table_size: int = 65537) -> list[int]:\n \"\"\"\n Generate a Maglev lookup table.\n table_size should be a prime number > len(backends).\n \"\"\"\n n = len(backends)\n\n # Generate permutation for each backend\n # offset and skip determine each backend's preferred positions\n permutations = []\n for backend in backends:\n h1 = hash(backend + \"_offset\") % table_size\n h2 = (hash(backend + \"_skip\") % (table_size - 1)) + 1\n perm = [(h1 + j * h2) % table_size for j in range(table_size)]\n permutations.append(perm)\n\n # Fill the table using the permutation preferences\n table = [-1] * table_size\n next_idx = [0] * n # Track each backend's next preferred position\n filled = 0\n\n while filled < table_size:\n for i in range(n):\n # Find backend i's next unfilled preferred position\n while table[permutations[i][next_idx[i]]] != -1:\n next_idx[i] += 1\n # Claim that position\n table[permutations[i][next_idx[i]]] = i\n next_idx[i] += 1\n filled += 1\n if filled == table_size:\n break\n\n return table\n\n# Usage\nbackends = [\"web-1\", \"web-2\", \"web-3\", \"web-4\", \"web-5\"]\ntable = maglev_table(backends)\n\n# Lookup: hash the request key, index into the table\ndef get_backend(key: str, table: list[int], backends: list[str]) -> str:\n idx = hash(key) % len(table)\n return backends[table[idx]]\n\n# When web-3 is removed, regenerate the table\n# Only ~1/n of entries change (compared to modular hashing where everything shifts)\nremaining = [\"web-1\", \"web-2\", \"web-4\", \"web-5\"]\nnew_table = maglev_table(remaining)\n```\n\nMaglev's advantage over ring-based consistent hashing: nearly perfect uniformity. With virtual nodes, ring-based hashing still has variance. Maglev fills a lookup table round-robin style per backend, so the distribution is almost exactly even."},{"heading":"Power of Two Choices (P2C)","level":2,"url":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing#power-of-two-choices-p2c","content":"P2C is elegantly simple: instead of picking one random backend, pick two random backends and send the request to whichever has fewer active connections.\n\nThe theoretical result is remarkable: random selection gives O(log n / log log n) max load, but P2C gives O(log log n) max load. That extra random choice exponentially reduces the worst case.\n\n```python\n# p2c.py - Power of Two Choices\nimport random\n\nclass P2CBalancer:\n def __init__(self, backends: list[str]):\n self.backends = backends\n self.active_connections = {b: 0 for b in backends}\n\n def pick(self) -> str:\n \"\"\"Pick the less-loaded of two random backends.\"\"\"\n if len(self.backends) < 2:\n return self.backends[0]\n\n a, b = random.sample(self.backends, 2)\n\n if self.active_connections[a] <= self.active_connections[b]:\n chosen = a\n else:\n chosen = b\n\n self.active_connections[chosen] += 1\n return chosen\n\n def release(self, backend: str):\n self.active_connections[backend] = max(0, self.active_connections[backend] - 1)\n```"},{"heading":"Real Production Configurations","level":2,"url":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing#real-production-configurations","content":""},{"heading":"HAProxy","level":3,"url":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing#haproxy","content":"```haproxy\n# haproxy.cfg - Production load balancing configuration\n\nglobal\n maxconn 50000\n log stdout local0\n\ndefaults\n mode http\n timeout connect 5s\n timeout client 30s\n timeout server 30s\n option httplog\n option forwardfor\n\n# Backend with least connections - for API with variable latency\nbackend api_servers\n balance leastconn\n option httpchk GET /health\n http-check expect status 200\n\n # Slow start: new backends ramp up over 30 seconds\n default-server inter 3s fall 3 rise 2 slowstart 30s\n\n server api-1 10.0.1.10:8080 check weight 100\n server api-2 10.0.1.11:8080 check weight 100\n server api-3 10.0.1.12:8080 check weight 100\n server api-4 10.0.1.13:8080 check weight 50 # Smaller instance\n\n# Backend with consistent hashing + bounded loads - for cache affinity\nbackend cache_servers\n balance uri\n hash-type consistent sdbm\n hash-balance-factor 150 # No backend gets >150% of average\n\n option httpchk GET /health\n http-check expect status 200\n\n server cache-1 10.0.2.10:8080 check\n server cache-2 10.0.2.11:8080 check\n server cache-3 10.0.2.12:8080 check\n server cache-4 10.0.2.13:8080 check\n\n# Backend with P2C - for general workloads\nbackend web_servers\n balance random(2) # Power of two choices\n\n option httpchk GET /health\n http-check expect status 200\n\n server web-1 10.0.3.10:8080 check\n server web-2 10.0.3.11:8080 check\n server web-3 10.0.3.12:8080 check\n\nfrontend http_front\n bind *:80\n bind *:443 ssl crt /etc/ssl/cert.pem\n\n acl is_api path_beg /api/\n acl is_cached path_beg /products/ /categories/\n\n use_backend api_servers if is_api\n use_backend cache_servers if is_cached\n default_backend web_servers\n```"},{"heading":"Envoy","level":3,"url":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing#envoy","content":"```yaml\n# envoy.yaml - Envoy proxy with multiple LB policies\nstatic_resources:\n clusters:\n - name: api_cluster\n type: STRICT_DNS\n lb_policy: LEAST_REQUEST\n least_request_lb_config:\n choice_count: 3 # P3C - pick best of 3\n health_checks:\n - timeout: 2s\n interval: 5s\n unhealthy_threshold: 3\n healthy_threshold: 2\n http_health_check:\n path: /health\n load_assignment:\n cluster_name: api_cluster\n endpoints:\n - lb_endpoints:\n - endpoint:\n address:\n socket_address:\n address: api-1.internal\n port_value: 8080\n - endpoint:\n address:\n socket_address:\n address: api-2.internal\n port_value: 8080\n\n - name: cache_cluster\n type: STRICT_DNS\n lb_policy: RING_HASH\n ring_hash_lb_config:\n minimum_ring_size: 1024\n maximum_ring_size: 8388608\n hash_function: XX_HASH\n health_checks:\n - timeout: 2s\n interval: 5s\n http_health_check:\n path: /health\n load_assignment:\n cluster_name: cache_cluster\n endpoints:\n - lb_endpoints:\n - endpoint:\n address:\n socket_address:\n address: cache-1.internal\n port_value: 8080\n - endpoint:\n address:\n socket_address:\n address: cache-2.internal\n port_value: 8080\n\n - name: websocket_cluster\n type: STRICT_DNS\n lb_policy: LEAST_REQUEST\n common_lb_config:\n healthy_panic_threshold:\n value: 50 # If <50% backends healthy, use all\n load_assignment:\n cluster_name: websocket_cluster\n endpoints:\n - lb_endpoints:\n - endpoint:\n address:\n socket_address:\n address: ws-1.internal\n port_value: 8080\n load_balancing_weight: 100\n - endpoint:\n address:\n socket_address:\n address: ws-2.internal\n port_value: 8080\n load_balancing_weight: 100\n```\n\nThe ` choice_count: 3 ` in Envoy's ` LEAST_REQUEST ` config gives you P3C, pick the best of three random backends. Research shows P3C gives marginal improvement over P2C, but it's free in Envoy so why not.\n\nThe ` healthy_panic_threshold ` for WebSocket backends is critical. Without it, if two of four backends fail health checks, Envoy only sends traffic to the two healthy ones, which might overload them. With the panic threshold at 50%, if more than half the backends fail, Envoy sends traffic to all of them (including \"unhealthy\" ones) on the theory that partial service is better than cascading failure."},{"heading":"Practical Recommendations","level":2,"url":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing#practical-recommendations","content":"After running these algorithms in production across different workloads, the following defaults emerge:\n\n- **Stateless HTTP APIs**: ` leastconn ` (HAProxy) or ` LEAST_REQUEST ` with ` choice_count: 2 ` (Envoy). Simple, effective, handles variable latency gracefully.\n- **Cache layers**: Consistent hashing with bounded loads. HAProxy's ` hash-balance-factor ` or a custom implementation. Cache hit rates matter enough to justify the complexity.\n- **WebSocket/gRPC streams**: Least connections, always. Long-lived connections need to balance by connection count, not by request count.\n- **General web traffic**: P2C. Almost as good as least connections, but without the overhead of tracking connection counts across all proxies in a distributed setup.\n\nP2C is perhaps the most impactful algorithm in this comparison. The realization that one extra random choice can exponentially reduce worst-case load is a powerful insight. Using ` random(2) ` in HAProxy for any backend without a strong reason to use something else is a pragmatic default."},{"heading":"Frequently asked questions","level":2,"url":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing#questions","content":""},{"heading":"What load balancing algorithm should I use by default?","level":3,"url":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing#question-1","content":"Round-robin for stateless HTTP services. Least connections for services with variable request duration. Consistent hashing only when you need session affinity or cache locality. P2C (power of two choices) when you want simplicity with near-optimal load distribution."},{"heading":"What is Maglev hashing and when should I use it?","level":3,"url":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing#question-2","content":"Maglev is Google's consistent hashing algorithm designed for their network load balancer. It provides minimal disruption when backends change and extremely uniform distribution. Use it when you're building a custom L4 load balancer or when you need better distribution than ring-based consistent hashing."},{"heading":"How does consistent hashing with bounded loads work?","level":3,"url":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing#question-3","content":"It's consistent hashing with a cap: no backend can receive more than (1 + epsilon) times the average load. When a backend hits its bound, the request spills over to the next backend on the ring. This prevents hot spots while maintaining most of the consistency benefits."},{"heading":"Tips","level":2,"url":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing#article-tips","content":"1. **Tip:** Use ` leastconn ` for long-lived connections (WebSocket, gRPC streams) and ` roundrobin ` for short HTTP requests. Mixing them up is the most common load balancing mistake.\n2. **Tip:** Envoy's ` RING_HASH ` with ` minimum_ring_size: 1024 ` gives you consistent hashing that survives backend additions without reshuffling all connections.\n3. **Tip:** Power of Two Choices (P2C) outperforms pure random selection dramatically. HAProxy's ` random(2) ` gives you P2C with zero configuration complexity.\n4. **Tip:** Always set ` hash-balance-factor ` in HAProxy when using consistent hashing. Without it, a hot key sends 100% of traffic to one backend. A factor of 150 caps any backend at 1.5x average load.\n\n- load-balancing\n- maglev\n- consistent-hashing\n- p2c\n- haproxy\n- envoy\n- infrastructure\n\n[Back to Blog](<https://lucioduran.com/blog>)"}],"links":[{"text":"Español","url":"https://lucioduran.com/es/blog/load-balancing-algorithms-maglev-consistent-hashing"},{"text":"HAProxy","url":"https://www.haproxy.org/"},{"text":"Envoy","url":"https://www.envoyproxy.io/"},{"text":"Back to Blog","url":"https://lucioduran.com/blog"}],"structuredData":[{"@context":"https://schema.org","@graph":[{"@type":"BlogPosting","@id":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing#article","mainEntityOfPage":{"@type":"WebPage","@id":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing"},"headline":"Load Balancing Algorithms: Maglev Hashing, Consistent Hashing with Bounded Loads, and P2C","description":"An analysis of modern load balancing algorithms, Google's Maglev hashing, consistent hashing with bounded loads, power of two choices, and production HAProxy/Envoy configurations.","datePublished":"2026-01-30","author":{"@type":"Person","@id":"https://lucioduran.com/#person","name":"Lucio Durán","url":"https://lucioduran.com"},"publisher":{"@type":"Person","@id":"https://lucioduran.com/#person","name":"Lucio Durán","url":"https://lucioduran.com"},"image":"https://lucioduran.com/favicons/og-default.png","url":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing","inLanguage":"en","isPartOf":{"@id":"https://lucioduran.com/blog#blog"},"articleSection":"Infrastructure","keywords":"load-balancing, maglev, consistent-hashing, p2c, haproxy, envoy, infrastructure","wordCount":1718,"timeRequired":"PT9M","workTranslation":[{"@type":"BlogPosting","inLanguage":"es","url":"https://lucioduran.com/es/blog/load-balancing-algorithms-maglev-consistent-hashing"}]},{"@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://lucioduran.com/"},{"@type":"ListItem","position":2,"name":"Blog","item":"https://lucioduran.com/blog"},{"@type":"ListItem","position":3,"name":"Load Balancing Algorithms: Maglev Hashing, Consistent Hashing with Bounded Loads, and P2C","item":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing"}]},{"@type":"FAQPage","mainEntity":[{"@type":"Question","name":"What load balancing algorithm should I use by default?","acceptedAnswer":{"@type":"Answer","text":"Round-robin for stateless HTTP services. Least connections for services with variable request duration. Consistent hashing only when you need session affinity or cache locality. P2C (power of two choices) when you want simplicity with near-optimal load distribution."}},{"@type":"Question","name":"What is Maglev hashing and when should I use it?","acceptedAnswer":{"@type":"Answer","text":"Maglev is Google's consistent hashing algorithm designed for their network load balancer. It provides minimal disruption when backends change and extremely uniform distribution. Use it when you're building a custom L4 load balancer or when you need better distribution than ring-based consistent hashing."}},{"@type":"Question","name":"How does consistent hashing with bounded loads work?","acceptedAnswer":{"@type":"Answer","text":"It's consistent hashing with a cap: no backend can receive more than (1 + epsilon) times the average load. When a backend hits its bound, the request spills over to the next backend on the ring. This prevents hot spots while maintaining most of the consistency benefits."}}]}]}],"contentHash":"c7f43206483eba0743d4ead2ae0b4e5d67e3850305027a9e175a658060749565","provenance":{"method":"Deterministic serialization of the server-rendered main element","source":"https://lucioduran.com/blog/load-balancing-algorithms-maglev-consistent-hashing","hashAlgorithm":"sha256","hashScope":"JSON document fields before contentHash and provenance","sourceHTMLBytes":67393,"contentBytes":15214}}