Databases # Valkey vs Redis: Migrating 14TB of Production Data, What the Documentation Doesn't Cover 2026-02-029 min read Lucio Durán Engineering Manager & AI Solutions Architect Also available in: [Español]() ## Why Valkey Exists (The Short Version) Redis Labs changed their license from BSD to a dual SSPL/RSAL license in March 2024. This made it impossible for cloud providers to offer managed Redis without a commercial agreement. [AWS](), Google, Oracle, and others backed the [Valkey]() fork under the Linux Foundation with a BSD-3-Clause license. Politics aside, the relevant question is whether the software is better. The following sections examine the actual engineering. ## RESP3: The Protocol Nobody Uses (But Should) RESP3 (Redis Serialization Protocol v3) was introduced in Redis 6 but barely anyone uses it because RESP2 still works and most client libraries default to it. This is a mistake. RESP3 introduces typed responses. In RESP2, everything is either a string, integer, array, or error. Your client library has to guess types: ``` # RESP2 response to HGETALL *4 $4 name $5 Lucio $3 age $2 38 # Client must know that alternating elements form key-value pairs # and that "38" should probably be an integer ``` ``` # RESP3 response to HGETALL %2 $4 name $5 Lucio $3 age :38 # The % prefix means "map with 2 entries" # The : prefix means "this is an integer" # No guessing needed ``` The practical impact is bigger than it looks. In Node.js services using ioredis, switching to RESP3 eliminates a pile of ` parseInt() ` and ` JSON.parse() ` calls on the application side. CPU utilization on the app servers can drop by ~8% on the hot path, not because the protocol is faster over the wire, but because we stopped doing redundant type coercion. To enable RESP3 on an existing connection: ```python # Python with valkey-py (fork of redis-py) import valkey client = valkey.Valkey( host='valkey-primary.internal', port=6379, protocol=3 # This sends HELLO 3 on connect ) # Now responses are natively typed result = client.hgetall('user:1001') # Returns: {'name': 'Lucio', 'age': 38} # age is already an int, not b'38' ``` ## Multi-Threaded I/O: Where Valkey Actually Diverges Redis has had ` io-threads ` since version 6, but it was limited to write operations and felt like an afterthought. Valkey took this further with both read and write threading, and the implementation is genuinely different. The architecture: ``` Main Thread ┌──────────┐ │ Command │ │ Execution│ │ (single) │ └────┬─────┘ │ ┌──────────┼──────────┐ │ │ │ ┌────┴───┐ ┌───┴────┐ ┌──┴─────┐ │ I/O │ │ I/O │ │ I/O │ │Thread 1│ │Thread 2│ │Thread 3│ │ read/ │ │ read/ │ │ read/ │ │ write │ │ write │ │ write │ └────────┘ └────────┘ └────────┘ ``` Command execution is still single-threaded (this is fundamental to Valkey/Redis's consistency model, no locking needed). But I/O parsing and response serialization happen in parallel across threads. Here is a configuration that works well on 8-core instances: ```conf # valkey.conf io-threads 4 io-threads-do-reads yes # Don't set io-threads higher than your core count minus 2 # Main thread + background tasks need CPU headroom # 4 threads on 8-core is typically the sweet spot # Increase the backlog for high-connection workloads tcp-backlog 4096 ``` This was benchmarked systematically using ` valkey-benchmark ` with different connection counts: ```bash # Single connection (no threading benefit expected) valkey-benchmark -h 127.0.0.1 -p 6379 -c 1 -n 1000000 \ -t set,get -d 256 --threads 1 # 200 connections (where threading matters) valkey-benchmark -h 127.0.0.1 -p 6379 -c 200 -n 5000000 \ -t set,get -d 256 --threads 4 # 1000 connections (stress test) valkey-benchmark -h 127.0.0.1 -p 6379 -c 1000 -n 10000000 \ -t set,get -d 256 --threads 8 ``` Results on an r7g.2xlarge (8 vCPU, 64GB): | Connections | Redis 7.2 (ops/s) | Valkey 8.0 1-thread (ops/s) | Valkey 8.0 4-thread (ops/s) | | --- | --- | --- | --- | | 1 | 142,000 | 144,000 | 143,000 | | 50 | 398,000 | 405,000 | 612,000 | | 200 | 445,000 | 451,000 | 891,000 | | 500 | 462,000 | 468,000 | 1,024,000 | | 1000 | 471,000 | 475,000 | 1,143,000 | At 1 connection, no difference. At 1000 connections, Valkey with 4 I/O threads delivers 2.4x the throughput. This isn't magic, it's simply that socket read/write and response serialization are parallelized while command execution remains serialized. ## The Migration: 14TB Across 12 Shards The setup: 12 Redis clusters (each a primary + 2 replicas) handling session data, feature flags, rate limiting, and a real-time analytics pipeline. Total dataset: ~14TB across all shards. The migration strategy was zero-downtime rolling replacement: ```bash #!/bin/bash # Simplified version of our migration script # Real version had error handling, rollback, and monitoring hooks SHARD=$1 REDIS_HOST="redis-${SHARD}.internal" VALKEY_HOST="valkey-${SHARD}.internal" # Step 1: Configure Valkey as replica of Redis valkey-cli -h $VALKEY_HOST \ REPLICAOF $REDIS_HOST 6379 # Step 2: Wait for sync to complete while true; do STATUS=$(valkey-cli -h $VALKEY_HOST INFO replication \ | grep master_sync_in_progress) if [[ $STATUS == *"0"* ]]; then echo "Sync complete for shard $SHARD" break fi sleep 5 done # Step 3: Verify data integrity REDIS_KEYS=$(redis-cli -h $REDIS_HOST DBSIZE | awk '{print $2}') VALKEY_KEYS=$(valkey-cli -h $VALKEY_HOST DBSIZE | awk '{print $2}') DIFF=$((VALKEY_KEYS - REDIS_KEYS)) if [ ${DIFF#-} -gt 100 ]; then echo "ERROR: Key count mismatch: Redis=$REDIS_KEYS Valkey=$VALKEY_KEYS" exit 1 fi # Step 4: Promote Valkey and switch traffic valkey-cli -h $VALKEY_HOST REPLICAOF NO ONE # Step 5: Update DNS/service discovery # (using Consul, this is a catalog update) consul kv put "cache/shard-${SHARD}/primary" "$VALKEY_HOST:6379" ``` The replication protocol between Redis and Valkey is compatible (Valkey can replicate from Redis and vice versa up to the fork point). This was the critical enabler for zero-downtime migration. The approach was one shard per day over two weeks, monitoring each for 24 hours before moving to the next. ### Gotchas Encountered **1. ` maxmemory-policy ` behavior difference.** Valkey's ` allkeys-lfu ` eviction has a slightly different frequency decay curve than Redis 7.2. The cache hit rate dropped by ~3% on one shard after migration. The fix was tuning ` lfu-decay-time ` from the default 1 to 3: ```conf lfu-decay-time 3 # Slower frequency decay = better at identifying truly hot keys # vs. keys that had a burst of access ``` **2. Lua script timeout handling.** A Lua script occasionally took \>5 seconds on large keyspaces. Redis kills the script and returns BUSY. Valkey also kills it, but the error message format is slightly different, which broke the retry logic that parsed the error string. Don't parse error messages. Use error codes. **3. RDB format version.** Valkey introduced RDB version 12 for some of its new data structures. If you need to fall back to Redis, you can't load a Valkey-generated RDB file that uses the new format. It is advisable to keep Redis-compatible RDB backups for the first month: ```conf # Force RDB compatibility during migration period rdb-compat-version 11 ``` ## Memory Optimization Analysis Valkey 8.0 introduced changes to how small data structures are stored. The ` listpack ` encoding (which replaced ` ziplist ` in Redis 7) got further optimized with better memory alignment and reduced per-entry overhead. Memory usage was measured on identical datasets: ```python # Test script: populate identical data structures import valkey import json import random import string def random_string(n): return ''.join(random.choices(string.ascii_letters, k=n)) def populate(client, prefix, count=1_000_000): pipe = client.pipeline() for i in range(count): # Mix of data types pipe.set(f'{prefix}:str:{i}', random_string(64)) pipe.hset(f'{prefix}:hash:{i}', mapping={ 'field1': random_string(32), 'field2': str(random.randint(0, 100000)), 'field3': random_string(16), }) pipe.zadd(f'{prefix}:zset:{i % 10000}', {random_string(12): random.random()}) if i % 5000 == 0: pipe.execute() pipe = client.pipeline() pipe.execute() ``` Memory usage for 1M entries of each type: | Data Type | Redis 7.2 | Valkey 8.0 | Savings | | --- | --- | --- | --- | | Strings (64B values) | 89.2 MB | 87.4 MB | 2.0% | | Hashes (3 fields) | 142.7 MB | 131.3 MB | 8.0% | | Sorted Sets (100 members) | 167.4 MB | 158.9 MB | 5.1% | | Streams (1K entries) | 203.1 MB | 189.6 MB | 6.6% | The hash savings are the most impactful because listpack encoding handles the small-hash case better. The ` hash-max-listpack-entries ` default is 128 in both, but Valkey's listpack implementation packs entries more tightly. The active defragmentation is also noticeably better. After 48 hours of sustained writes and deletes on a test instance: ```bash # Check fragmentation ratio valkey-cli INFO memory | grep mem_fragmentation_ratio # Redis 7.2 after 48h: mem_fragmentation_ratio:1.43 # Valkey 8.0 after 48h: mem_fragmentation_ratio:1.12 ``` A fragmentation ratio of 1.43 means Redis is using 43% more memory than the actual data requires. Valkey kept it at 12%. On a 14TB dataset, that is the difference between needing 20TB of RAM and needing 15.7TB. At cloud prices, that is real money, roughly $2,400/month. ## Client Library State This matters more than most articles admit. The client library ecosystem determines whether migration is trivial or painful. | Language | Library | Status | | --- | --- | --- | | Python | valkey-py | Production-ready, fork of redis-py | | Node.js | iovalkey | Production-ready, fork of ioredis | | Java | Jedis 5.x | Native Valkey support | | Go | go-valkey | Production-ready, Valkey-first design | | Rust | valkey-rs | Based on redis-rs, works | | .NET | StackExchange.Redis | Works with Valkey (protocol-compatible) | Using iovalkey in Node.js, the migration is a package swap and config change: ```javascript // Before import Redis from 'ioredis'; const client = new Redis({ host: 'redis-primary.internal', port: 6379, maxRetriesPerRequest: 3, }); // After import { Valkey } from 'iovalkey'; const client = new Valkey({ host: 'valkey-primary.internal', port: 6379, maxRetriesPerRequest: 3, // Enable RESP3 for typed responses protocol: 3, }); ``` ## Should You Migrate? Here is an honest assessment after eight months of production use: **Migrate if:** You're running Redis OSS, you want multi-threaded I/O for high-connection workloads, the SSPL license concerns you, or you want better memory efficiency on hash-heavy workloads. **Don't migrate if:** You depend on Redis Stack modules (Search, JSON, TimeSeries, Graph), you're on Redis Enterprise with SLAs you trust, or your workload is single-connection (you won't see a difference). **Wait if:** You're not sure. Valkey isn't going anywhere, the Linux Foundation backing ensures longevity, and the contributor base is growing. You can migrate in a year and the process will only be smoother. One important point: framing this as a political choice misses the mark. Valkey is technically superior for high-connection workloads, better memory efficiency, better threading, same compatibility. If Redis were still BSD-licensed and had these features, the recommendation would be Redis. But it does not, and Valkey does. That is the end of the analysis. ## Frequently asked questions ### Is Valkey a drop-in replacement for Redis? For Redis OSS 7.2.x, yes, protocol-compatible, same command set. But if you use Redis Stack modules (RedisJSON, RedisSearch, RedisTimeSeries), those are proprietary and not available in Valkey. Community alternatives exist but aren't identical. ### What's the performance difference between Valkey and Redis? With multi-threaded I/O enabled, Valkey handles 2-3x more operations per second on connection-heavy workloads. On single-connection benchmarks, performance is nearly identical. The gap increases with connection count. ### Should I migrate from Redis to Valkey? If you're on Redis OSS and concerned about licensing (Redis switched to SSPL/RSAL), yes. If you're on Redis Enterprise with support, the business case depends on your scale and whether you need Redis Stack modules. ## Tips 1. **Tip:** Before migrating, run ` valkey-cli --bigkeys --memkeys ` on your Redis dump. Valkey's multi-threaded I/O shines on many small keys but can actually regress on a few massive hash keys (\>50MB) due to thread contention on serialization. 2. **Tip:** Set ` io-threads 4 ` and ` io-threads-do-reads yes ` in your Valkey config. The default is single-threaded for backward compat, which means you get zero benefit over Redis out of the box. 3. **Tip:** If you're using RESP2 clients, switch to RESP3 with ` HELLO 3 `, the typed responses eliminate client-side type coercion, which alone saved us ~8% CPU on our application servers. 4. **Tip:** Use ` activedefrag yes ` with ` active-defrag-threshold-lower 10 ` in Valkey. Its defragmentation is measurably better than Redis 7.x, especially after sustained write-heavy workloads. - valkey - redis - in-memory-database - resp3 - performance - migration - multi-threaded-io - caching [Back to Blog]()