{"schemaVersion":1,"id":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency","url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency","title":"ScyllaDB: Cassandra-Compatible Low-Latency Database Architecture and Migration","description":"A deep dive into ScyllaDB's shard-per-core architecture, Seastar framework internals, and real p99 latency benchmarks against Apache Cassandra. Plus a battle-tested migration guide.","language":"en","datePublished":"2025-11-20","dateModified":null,"contentFormat":"text/markdown","content":"Databases\n\n# ScyllaDB: Cassandra-Compatible Low-Latency Database Architecture and Migration\n\n2025-11-208 min read\n\nLucio Durán\nEngineering Manager & AI Solutions Architect\n\nAlso available in: [Español](<https://lucioduran.com/es/blog/scylladb-cassandra-compatible-low-latency>)\n\n## The Problem with Cassandra's Architecture\n\nConsider what happens inside a Cassandra node. You've got a JVM process managing tens of gigabytes of heap, running on a machine with 16-32 cores. Every read and write goes through shared data structures protected by locks. The JVM's garbage collector periodically decides to pause everything, and \"everything\" means all 32 cores stop doing useful work simultaneously.\n\nHere is a typical Cassandra GC log entry that illustrates the problem:\n\n```\nGC pause (G1 Evacuation Pause) 14.2GB->9.8GB 247.483ms\n```\n\n247 milliseconds. That's not a tail latency outlier, that's a quarter second where your database simply doesn't respond. And this was on a tuned cluster with G1GC, 8GB new gen, and carefully sized regions.\n\nThe fundamental issue is that Cassandra's threading model was designed in the mid-2000s. It uses a thread-per-core approach with shared mutable state, and then layers a garbage-collected runtime on top. Every architectural choice compounds:\n\n```\nThread pool executor (WRITE stage) → Shared memtable → Lock contention\nThread pool executor (READ stage) → Shared row cache → Lock contention\nJVM GC → Stop-the-world → All threads pause → Latency spike\n```\n\n## Seastar: The Shard-Per-Core Framework\n\n[ScyllaDB](<https://www.scylladb.com/>) is built on the Seastar framework, which was originally developed for high-performance networking (it powered DPDK-based applications before ScyllaDB existed). Seastar implements a **shard-per-core** architecture that eliminates shared state entirely.\n\nHere's the mental model. On a 16-core machine:\n\n```\nCore 0: [Shard 0] → own memtable, own cache, own I/O queue, own network connections\nCore 1: [Shard 1] → own memtable, own cache, own I/O queue, own network connections\nCore 2: [Shard 2] → own memtable, own cache, own I/O queue, own network connections\n...\nCore 15: [Shard 15] → own memtable, own cache, own I/O queue, own network connections\n```\n\nEach shard is a complete mini-database. There are **no locks**, because there's nothing shared to lock. When data needs to move between shards (which happens during range scans or when a request arrives at the wrong shard), Seastar uses explicit message passing, not shared memory.\n\nThe Seastar reactor loop on each core looks roughly like this:\n\n```cpp\nvoid reactor::run() {\n while (true) {\n // Poll for completed I/O (no syscall if using io_uring or AIO)\n poll_io();\n\n // Run ready tasks from this shard's task queue\n run_tasks();\n\n // Poll network (epoll or DPDK, depending on config)\n poll_network();\n\n // Check timers\n expire_timers();\n }\n}\n```\n\nThis is cooperative multitasking. There are no context switches, no thread migrations, no scheduler overhead. A task runs until it voluntarily yields (typically at an I/O boundary), and then the next task on that shard picks up. The CPU cache stays hot because the same core always processes the same data.\n\n## Shard-Per-Core in Practice: What It Means for Queries\n\nWhen a CQL query arrives at a ScyllaDB node, the coordinator determines which shard owns the partition key using a hash:\n\n```cpp\nunsigned shard_of(const partition_key& pk) {\n auto token = partitioner().get_token(pk);\n return token.raw() % smp::count; // simplified\n}\n```\n\nIf the request lands on the correct shard's core, processing is entirely local, no cross-core communication, no cache line bouncing, no NUMA penalties. The shard-aware drivers that ScyllaDB provides for Java, Go, Python, and Rust implement this same hashing client-side:\n\n```python\n# ScyllaDB Python shard-aware driver\nfrom cassandra.cluster import Cluster\nfrom cassandra.policies import TokenAwarePolicy, DCAwareRoundRobinPolicy\n\ncluster = Cluster(\n ['scylla-node-1', 'scylla-node-2', 'scylla-node-3'],\n load_balancing_policy=TokenAwarePolicy(DCAwareRoundRobinPolicy()),\n # The shard-aware extension adds shard routing on top of token awareness\n shard_aware_options=ShardAwareOptions()\n)\n```\n\nWith shard-aware routing, the driver opens one connection per shard per node. On a 3-node cluster with 16 cores each, that's 48 connections, but each connection goes directly to the right shard, eliminating the coordinator hop overhead.\n\n## Real p99 Latency Comparison\n\nThese benchmarks ran on identical hardware, 3 nodes, each with 16 vCPUs (c5d.4xlarge on [AWS](<https://aws.com/>)), 32GB RAM, NVMe local storage. Same schema, same dataset (800M partitions, ~2TB), same YCSB workload.\n\n```\nWorkload B (95% read, 5% write), 50K ops/sec sustained\n\n p50 p95 p99 p99.9\nCassandra 4.1 1.2ms 4.8ms 28.3ms 312ms\nScyllaDB 5.4 0.4ms 0.9ms 1.8ms 4.2ms\n```\n\nLook at p99.9. That 312ms on Cassandra? That's GC. There's nothing you can tune to make it go away, it's fundamental to the architecture. ScyllaDB's 4.2ms at p99.9 is I/O scheduling jitter, and it's predictable.\n\nThe write path is even more dramatic because Cassandra's memtable is a shared ConcurrentSkipListMap that becomes contended under write-heavy loads:\n\n```\nWorkload A (50% read, 50% write), 50K ops/sec sustained\n\n p50 p95 p99 p99.9\nCassandra 4.1 1.8ms 8.2ms 45.1ms 580ms\nScyllaDB 5.4 0.3ms 0.7ms 1.4ms 3.8ms\n```\n\n## The Compaction Story\n\nCassandra ships with SizeTieredCompactionStrategy (STCS) and LeveledCompactionStrategy (LCS). Both have well-known problems: STCS causes temporary space amplification up to 2x, LCS causes write amplification that kills throughput.\n\nScyllaDB introduced Incremental Compaction Strategy (ICS), which is their default and has no direct Cassandra equivalent. It operates at a sub-SSTable granularity, instead of reading and rewriting entire SSTables, it works on sorted runs within SSTables:\n\n```\nTraditional Compaction (STCS):\n SSTable_A (4GB) + SSTable_B (4GB) → SSTable_C (8GB)\n Temporary space needed: 16GB (input + output)\n\nIncremental Compaction (ICS):\n SSTable_A[run_1] + SSTable_B[run_1] → merged_run_1\n SSTable_A[run_2] + SSTable_B[run_2] → merged_run_2\n Temporary space needed: ~1GB (one run at a time)\n```\n\nThis means you can run ScyllaDB on 50% disk utilization instead of needing 50% free for compaction headroom. On a 2TB NVMe drive, that's an extra terabyte of usable storage.\n\n## Migration Guide: Cassandra to ScyllaDB\n\nHere is a proven migration process, validated across multiple production environments. It is less painful than expected.\n\n### Step 1: Schema Compatibility Check\n\nExport your Cassandra schema and validate it against ScyllaDB:\n\n```bash\ncqlsh cassandra-node -e \"DESCRIBE KEYSPACE my_keyspace\" > schema.cql\n\n# ScyllaDB supports 99% of CQL. Main incompatibilities:\n# - Materialized Views: supported but with different consistency guarantees\n# - SASI indexes: not supported, use ScyllaDB's secondary index implementation\n# - Custom types from Java: need equivalent UDTs\n```\n\n### Step 2: SSTable Streaming\n\nFor datasets under 2TB, direct SSTable loading is the fastest path:\n\n```bash\n# On each Cassandra node, take a snapshot\nnodetool snapshot -t migration_snap my_keyspace\n\n# Copy SSTables to ScyllaDB nodes (respecting token ranges)\n# ScyllaDB's sstableloader handles the token mapping\nsstableloader -d scylla-node-1,scylla-node-2,scylla-node-3 \\\n /var/lib/cassandra/data/my_keyspace/my_table-<uuid>/snapshots/migration_snap/\n```\n\n### Step 3: Validation\n\n```python\n# Compare row counts and checksums between clusters\nimport hashlib\nfrom cassandra.cluster import Cluster\n\ndef checksum_partition_range(session, table, token_start, token_end):\n rows = session.execute(f\"\"\"\n SELECT * FROM {table}\n WHERE token(partition_key) >= {token_start}\n AND token(partition_key) < {token_end}\n \"\"\")\n h = hashlib.sha256()\n count = 0\n for row in rows:\n h.update(str(row).encode())\n count += 1\n return count, h.hexdigest()\n```\n\n### Step 4: Traffic Cutover\n\nA shadow-traffic phase is strongly recommended where you send reads to both clusters and compare results:\n\n```python\nasync def dual_read(cassandra_session, scylla_session, query, params):\n cass_future = cassandra_session.execute_async(query, params)\n scylla_future = scylla_session.execute_async(query, params)\n\n cass_result = cass_future.result()\n scylla_result = scylla_future.result()\n\n if cass_result != scylla_result:\n log.error(f\"Mismatch for query {query}: cassandra={cass_result}, scylla={scylla_result}\")\n metrics.increment(\"dual_read.mismatch\")\n\n return scylla_result # serve from ScyllaDB when confident\n```\n\n## Operational Differences to Know\n\nA few important operational differences:\n\n**Memory management**: ScyllaDB uses all available RAM by default for its row cache and doesn't need heap tuning. Don't set memory limits unless you're sharing the machine (which is not recommended, see the tip above).\n\n**Workload prioritization**: ScyllaDB supports service levels that let you prioritize workloads per CQL user. This is incredible for multi-tenant setups:\n\n```sql\nCREATE SERVICE LEVEL gold WITH timeout = 500ms AND workload_type = 'interactive';\nCREATE SERVICE LEVEL batch WITH timeout = 30s AND workload_type = 'batch';\n\nALTER ROLE analytics_user WITH SERVICE LEVEL batch;\nALTER ROLE api_user WITH SERVICE LEVEL gold;\n```\n\n**Repair**: ScyllaDB's row-level repair is significantly faster than Cassandra's. What took 8 hours on a comparable Cassandra cluster finishes in 45 minutes on ScyllaDB, because it only streams the actual differing rows instead of entire SSTables.\n\n## When NOT to Use ScyllaDB\n\nScyllaDB is not the right choice for:\n\n- Datasets under 100GB where PostgreSQL is perfectly adequate\n- Workloads that are 99% full-table scans (use ClickHouse or DuckDB instead)\n- Teams that need the Apache Cassandra ecosystem of tools without modification\n\nBut if you're running Cassandra today and fighting with tail latencies, paying for extra nodes to compensate for GC pauses, or spending too much time tuning JVM parameters, ScyllaDB is the answer that addresses these problems directly.\n\nThe migration is not free, but teams consistently come out the other side running fewer nodes, with better latencies, and with significantly less operational toil. That is not marketing, that is production reality.\n\n## Frequently asked questions\n\n### Can I use my existing Cassandra drivers with ScyllaDB?\n\nYes. ScyllaDB implements the CQL native protocol, so any Cassandra driver works out of the box. However, ScyllaDB's shard-aware drivers (available for Java, Go, Python, Rust, and C++) route requests directly to the correct shard, cutting cross-shard coordination and reducing p99 latency by 30-60%.\n\n### How does ScyllaDB achieve lower tail latency than Cassandra?\n\nThree mechanisms: shard-per-core architecture eliminates lock contention, the Seastar framework uses cooperative scheduling with no context switches, and the I/O scheduler provides per-shard fairness. There's no garbage collector, no stop-the-world pauses, no JVM safepoints.\n\n### What's the real-world migration effort from a 12-node Cassandra cluster?\n\nTypically 3-6 weeks for a production migration including validation. The schema is nearly 1:1 compatible. Main effort goes into tuning compaction strategies (ScyllaDB's Incremental Compaction Strategy has no Cassandra equivalent) and adjusting client-side timeouts downward since ScyllaDB responds faster.\n\n## Tips\n\n1. **Tip:** When migrating from Cassandra, run ` nodetool snapshot ` and stream SSTables directly into ScyllaDB using ` sstableloader `. Skip the dual-write phase if your dataset is under 2TB, it's not worth the operational overhead.\n2. **Tip:** Set ` --overprovisioned ` flag to ` false ` in production and let ScyllaDB own the entire machine. Sharing a box with other workloads defeats the entire shard-per-core model and you'll get random latency spikes that make you question your life choices.\n3. **Tip:** Use ` scylla-monitoring-stack ` from day one. The Grafana dashboards ship with per-shard metrics that Cassandra simply cannot surface, and they'll save you weeks of debugging when you hit compaction storms.\n\n- scylladb\n- cassandra\n- distributed-databases\n- low-latency\n- seastar\n- cpp\n- nosql\n- shard-per-core\n\n[Back to Blog](<https://lucioduran.com/blog>)","headings":[{"level":1,"text":"ScyllaDB: Cassandra-Compatible Low-Latency Database Architecture and Migration","url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#title"},{"level":2,"text":"The Problem with Cassandra's Architecture","url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#the-problem-with-cassandras-architecture"},{"level":2,"text":"Seastar: The Shard-Per-Core Framework","url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#seastar-the-shard-per-core-framework"},{"level":2,"text":"Shard-Per-Core in Practice: What It Means for Queries","url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#shard-per-core-in-practice-what-it-means-for-queries"},{"level":2,"text":"Real p99 Latency Comparison","url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#real-p99-latency-comparison"},{"level":2,"text":"The Compaction Story","url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#the-compaction-story"},{"level":2,"text":"Migration Guide: Cassandra to ScyllaDB","url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#migration-guide-cassandra-to-scylladb"},{"level":3,"text":"Step 1: Schema Compatibility Check","url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#step-1-schema-compatibility-check"},{"level":3,"text":"Step 2: SSTable Streaming","url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#step-2-sstable-streaming"},{"level":3,"text":"Step 3: Validation","url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#step-3-validation"},{"level":3,"text":"Step 4: Traffic Cutover","url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#step-4-traffic-cutover"},{"level":2,"text":"Operational Differences to Know","url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#operational-differences-to-know"},{"level":2,"text":"When NOT to Use ScyllaDB","url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#when-not-to-use-scylladb"},{"level":2,"text":"Frequently asked questions","url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#questions"},{"level":3,"text":"Can I use my existing Cassandra drivers with ScyllaDB?","url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#question-1"},{"level":3,"text":"How does ScyllaDB achieve lower tail latency than Cassandra?","url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#question-2"},{"level":3,"text":"What's the real-world migration effort from a 12-node Cassandra cluster?","url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#question-3"},{"level":2,"text":"Tips","url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#article-tips"}],"sections":[{"heading":"ScyllaDB: Cassandra-Compatible Low-Latency Database Architecture and Migration","level":1,"url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency","content":"Databases"},{"heading":"ScyllaDB: Cassandra-Compatible Low-Latency Database Architecture and Migration","level":1,"url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#title","content":"2025-11-208 min read\n\nLucio Durán\nEngineering Manager & AI Solutions Architect\n\nAlso available in: [Español](<https://lucioduran.com/es/blog/scylladb-cassandra-compatible-low-latency>)"},{"heading":"The Problem with Cassandra's Architecture","level":2,"url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#the-problem-with-cassandras-architecture","content":"Consider what happens inside a Cassandra node. You've got a JVM process managing tens of gigabytes of heap, running on a machine with 16-32 cores. Every read and write goes through shared data structures protected by locks. The JVM's garbage collector periodically decides to pause everything, and \"everything\" means all 32 cores stop doing useful work simultaneously.\n\nHere is a typical Cassandra GC log entry that illustrates the problem:\n\n```\nGC pause (G1 Evacuation Pause) 14.2GB->9.8GB 247.483ms\n```\n\n247 milliseconds. That's not a tail latency outlier, that's a quarter second where your database simply doesn't respond. And this was on a tuned cluster with G1GC, 8GB new gen, and carefully sized regions.\n\nThe fundamental issue is that Cassandra's threading model was designed in the mid-2000s. It uses a thread-per-core approach with shared mutable state, and then layers a garbage-collected runtime on top. Every architectural choice compounds:\n\n```\nThread pool executor (WRITE stage) → Shared memtable → Lock contention\nThread pool executor (READ stage) → Shared row cache → Lock contention\nJVM GC → Stop-the-world → All threads pause → Latency spike\n```"},{"heading":"Seastar: The Shard-Per-Core Framework","level":2,"url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#seastar-the-shard-per-core-framework","content":"[ScyllaDB](<https://www.scylladb.com/>) is built on the Seastar framework, which was originally developed for high-performance networking (it powered DPDK-based applications before ScyllaDB existed). Seastar implements a **shard-per-core** architecture that eliminates shared state entirely.\n\nHere's the mental model. On a 16-core machine:\n\n```\nCore 0: [Shard 0] → own memtable, own cache, own I/O queue, own network connections\nCore 1: [Shard 1] → own memtable, own cache, own I/O queue, own network connections\nCore 2: [Shard 2] → own memtable, own cache, own I/O queue, own network connections\n...\nCore 15: [Shard 15] → own memtable, own cache, own I/O queue, own network connections\n```\n\nEach shard is a complete mini-database. There are **no locks**, because there's nothing shared to lock. When data needs to move between shards (which happens during range scans or when a request arrives at the wrong shard), Seastar uses explicit message passing, not shared memory.\n\nThe Seastar reactor loop on each core looks roughly like this:\n\n```cpp\nvoid reactor::run() {\n while (true) {\n // Poll for completed I/O (no syscall if using io_uring or AIO)\n poll_io();\n\n // Run ready tasks from this shard's task queue\n run_tasks();\n\n // Poll network (epoll or DPDK, depending on config)\n poll_network();\n\n // Check timers\n expire_timers();\n }\n}\n```\n\nThis is cooperative multitasking. There are no context switches, no thread migrations, no scheduler overhead. A task runs until it voluntarily yields (typically at an I/O boundary), and then the next task on that shard picks up. The CPU cache stays hot because the same core always processes the same data."},{"heading":"Shard-Per-Core in Practice: What It Means for Queries","level":2,"url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#shard-per-core-in-practice-what-it-means-for-queries","content":"When a CQL query arrives at a ScyllaDB node, the coordinator determines which shard owns the partition key using a hash:\n\n```cpp\nunsigned shard_of(const partition_key& pk) {\n auto token = partitioner().get_token(pk);\n return token.raw() % smp::count; // simplified\n}\n```\n\nIf the request lands on the correct shard's core, processing is entirely local, no cross-core communication, no cache line bouncing, no NUMA penalties. The shard-aware drivers that ScyllaDB provides for Java, Go, Python, and Rust implement this same hashing client-side:\n\n```python\n# ScyllaDB Python shard-aware driver\nfrom cassandra.cluster import Cluster\nfrom cassandra.policies import TokenAwarePolicy, DCAwareRoundRobinPolicy\n\ncluster = Cluster(\n ['scylla-node-1', 'scylla-node-2', 'scylla-node-3'],\n load_balancing_policy=TokenAwarePolicy(DCAwareRoundRobinPolicy()),\n # The shard-aware extension adds shard routing on top of token awareness\n shard_aware_options=ShardAwareOptions()\n)\n```\n\nWith shard-aware routing, the driver opens one connection per shard per node. On a 3-node cluster with 16 cores each, that's 48 connections, but each connection goes directly to the right shard, eliminating the coordinator hop overhead."},{"heading":"Real p99 Latency Comparison","level":2,"url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#real-p99-latency-comparison","content":"These benchmarks ran on identical hardware, 3 nodes, each with 16 vCPUs (c5d.4xlarge on [AWS](<https://aws.com/>)), 32GB RAM, NVMe local storage. Same schema, same dataset (800M partitions, ~2TB), same YCSB workload.\n\n```\nWorkload B (95% read, 5% write), 50K ops/sec sustained\n\n p50 p95 p99 p99.9\nCassandra 4.1 1.2ms 4.8ms 28.3ms 312ms\nScyllaDB 5.4 0.4ms 0.9ms 1.8ms 4.2ms\n```\n\nLook at p99.9. That 312ms on Cassandra? That's GC. There's nothing you can tune to make it go away, it's fundamental to the architecture. ScyllaDB's 4.2ms at p99.9 is I/O scheduling jitter, and it's predictable.\n\nThe write path is even more dramatic because Cassandra's memtable is a shared ConcurrentSkipListMap that becomes contended under write-heavy loads:\n\n```\nWorkload A (50% read, 50% write), 50K ops/sec sustained\n\n p50 p95 p99 p99.9\nCassandra 4.1 1.8ms 8.2ms 45.1ms 580ms\nScyllaDB 5.4 0.3ms 0.7ms 1.4ms 3.8ms\n```"},{"heading":"The Compaction Story","level":2,"url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#the-compaction-story","content":"Cassandra ships with SizeTieredCompactionStrategy (STCS) and LeveledCompactionStrategy (LCS). Both have well-known problems: STCS causes temporary space amplification up to 2x, LCS causes write amplification that kills throughput.\n\nScyllaDB introduced Incremental Compaction Strategy (ICS), which is their default and has no direct Cassandra equivalent. It operates at a sub-SSTable granularity, instead of reading and rewriting entire SSTables, it works on sorted runs within SSTables:\n\n```\nTraditional Compaction (STCS):\n SSTable_A (4GB) + SSTable_B (4GB) → SSTable_C (8GB)\n Temporary space needed: 16GB (input + output)\n\nIncremental Compaction (ICS):\n SSTable_A[run_1] + SSTable_B[run_1] → merged_run_1\n SSTable_A[run_2] + SSTable_B[run_2] → merged_run_2\n Temporary space needed: ~1GB (one run at a time)\n```\n\nThis means you can run ScyllaDB on 50% disk utilization instead of needing 50% free for compaction headroom. On a 2TB NVMe drive, that's an extra terabyte of usable storage."},{"heading":"Migration Guide: Cassandra to ScyllaDB","level":2,"url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#migration-guide-cassandra-to-scylladb","content":"Here is a proven migration process, validated across multiple production environments. It is less painful than expected."},{"heading":"Step 1: Schema Compatibility Check","level":3,"url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#step-1-schema-compatibility-check","content":"Export your Cassandra schema and validate it against ScyllaDB:\n\n```bash\ncqlsh cassandra-node -e \"DESCRIBE KEYSPACE my_keyspace\" > schema.cql\n\n# ScyllaDB supports 99% of CQL. Main incompatibilities:\n# - Materialized Views: supported but with different consistency guarantees\n# - SASI indexes: not supported, use ScyllaDB's secondary index implementation\n# - Custom types from Java: need equivalent UDTs\n```"},{"heading":"Step 2: SSTable Streaming","level":3,"url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#step-2-sstable-streaming","content":"For datasets under 2TB, direct SSTable loading is the fastest path:\n\n```bash\n# On each Cassandra node, take a snapshot\nnodetool snapshot -t migration_snap my_keyspace\n\n# Copy SSTables to ScyllaDB nodes (respecting token ranges)\n# ScyllaDB's sstableloader handles the token mapping\nsstableloader -d scylla-node-1,scylla-node-2,scylla-node-3 \\\n /var/lib/cassandra/data/my_keyspace/my_table-<uuid>/snapshots/migration_snap/\n```"},{"heading":"Step 3: Validation","level":3,"url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#step-3-validation","content":"```python\n# Compare row counts and checksums between clusters\nimport hashlib\nfrom cassandra.cluster import Cluster\n\ndef checksum_partition_range(session, table, token_start, token_end):\n rows = session.execute(f\"\"\"\n SELECT * FROM {table}\n WHERE token(partition_key) >= {token_start}\n AND token(partition_key) < {token_end}\n \"\"\")\n h = hashlib.sha256()\n count = 0\n for row in rows:\n h.update(str(row).encode())\n count += 1\n return count, h.hexdigest()\n```"},{"heading":"Step 4: Traffic Cutover","level":3,"url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#step-4-traffic-cutover","content":"A shadow-traffic phase is strongly recommended where you send reads to both clusters and compare results:\n\n```python\nasync def dual_read(cassandra_session, scylla_session, query, params):\n cass_future = cassandra_session.execute_async(query, params)\n scylla_future = scylla_session.execute_async(query, params)\n\n cass_result = cass_future.result()\n scylla_result = scylla_future.result()\n\n if cass_result != scylla_result:\n log.error(f\"Mismatch for query {query}: cassandra={cass_result}, scylla={scylla_result}\")\n metrics.increment(\"dual_read.mismatch\")\n\n return scylla_result # serve from ScyllaDB when confident\n```"},{"heading":"Operational Differences to Know","level":2,"url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#operational-differences-to-know","content":"A few important operational differences:\n\n**Memory management**: ScyllaDB uses all available RAM by default for its row cache and doesn't need heap tuning. Don't set memory limits unless you're sharing the machine (which is not recommended, see the tip above).\n\n**Workload prioritization**: ScyllaDB supports service levels that let you prioritize workloads per CQL user. This is incredible for multi-tenant setups:\n\n```sql\nCREATE SERVICE LEVEL gold WITH timeout = 500ms AND workload_type = 'interactive';\nCREATE SERVICE LEVEL batch WITH timeout = 30s AND workload_type = 'batch';\n\nALTER ROLE analytics_user WITH SERVICE LEVEL batch;\nALTER ROLE api_user WITH SERVICE LEVEL gold;\n```\n\n**Repair**: ScyllaDB's row-level repair is significantly faster than Cassandra's. What took 8 hours on a comparable Cassandra cluster finishes in 45 minutes on ScyllaDB, because it only streams the actual differing rows instead of entire SSTables."},{"heading":"When NOT to Use ScyllaDB","level":2,"url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#when-not-to-use-scylladb","content":"ScyllaDB is not the right choice for:\n\n- Datasets under 100GB where PostgreSQL is perfectly adequate\n- Workloads that are 99% full-table scans (use ClickHouse or DuckDB instead)\n- Teams that need the Apache Cassandra ecosystem of tools without modification\n\nBut if you're running Cassandra today and fighting with tail latencies, paying for extra nodes to compensate for GC pauses, or spending too much time tuning JVM parameters, ScyllaDB is the answer that addresses these problems directly.\n\nThe migration is not free, but teams consistently come out the other side running fewer nodes, with better latencies, and with significantly less operational toil. That is not marketing, that is production reality."},{"heading":"Frequently asked questions","level":2,"url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#questions","content":""},{"heading":"Can I use my existing Cassandra drivers with ScyllaDB?","level":3,"url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#question-1","content":"Yes. ScyllaDB implements the CQL native protocol, so any Cassandra driver works out of the box. However, ScyllaDB's shard-aware drivers (available for Java, Go, Python, Rust, and C++) route requests directly to the correct shard, cutting cross-shard coordination and reducing p99 latency by 30-60%."},{"heading":"How does ScyllaDB achieve lower tail latency than Cassandra?","level":3,"url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#question-2","content":"Three mechanisms: shard-per-core architecture eliminates lock contention, the Seastar framework uses cooperative scheduling with no context switches, and the I/O scheduler provides per-shard fairness. There's no garbage collector, no stop-the-world pauses, no JVM safepoints."},{"heading":"What's the real-world migration effort from a 12-node Cassandra cluster?","level":3,"url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#question-3","content":"Typically 3-6 weeks for a production migration including validation. The schema is nearly 1:1 compatible. Main effort goes into tuning compaction strategies (ScyllaDB's Incremental Compaction Strategy has no Cassandra equivalent) and adjusting client-side timeouts downward since ScyllaDB responds faster."},{"heading":"Tips","level":2,"url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#article-tips","content":"1. **Tip:** When migrating from Cassandra, run ` nodetool snapshot ` and stream SSTables directly into ScyllaDB using ` sstableloader `. Skip the dual-write phase if your dataset is under 2TB, it's not worth the operational overhead.\n2. **Tip:** Set ` --overprovisioned ` flag to ` false ` in production and let ScyllaDB own the entire machine. Sharing a box with other workloads defeats the entire shard-per-core model and you'll get random latency spikes that make you question your life choices.\n3. **Tip:** Use ` scylla-monitoring-stack ` from day one. The Grafana dashboards ship with per-shard metrics that Cassandra simply cannot surface, and they'll save you weeks of debugging when you hit compaction storms.\n\n- scylladb\n- cassandra\n- distributed-databases\n- low-latency\n- seastar\n- cpp\n- nosql\n- shard-per-core\n\n[Back to Blog](<https://lucioduran.com/blog>)"}],"links":[{"text":"Español","url":"https://lucioduran.com/es/blog/scylladb-cassandra-compatible-low-latency"},{"text":"ScyllaDB","url":"https://www.scylladb.com/"},{"text":"AWS","url":"https://aws.com/"},{"text":"Back to Blog","url":"https://lucioduran.com/blog"}],"structuredData":[{"@context":"https://schema.org","@graph":[{"@type":"BlogPosting","@id":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency#article","mainEntityOfPage":{"@type":"WebPage","@id":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency"},"headline":"ScyllaDB: Cassandra-Compatible Low-Latency Database Architecture and Migration","description":"A deep dive into ScyllaDB's shard-per-core architecture, Seastar framework internals, and real p99 latency benchmarks against Apache Cassandra. Plus a battle-tested migration guide.","datePublished":"2025-11-20","author":{"@type":"Person","@id":"https://lucioduran.com/#person","name":"Lucio Durán","url":"https://lucioduran.com"},"publisher":{"@type":"Person","@id":"https://lucioduran.com/#person","name":"Lucio Durán","url":"https://lucioduran.com"},"image":"https://statics.lucioduran.com/images/lucioduran-social-card.jpg","url":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency","inLanguage":"en","isPartOf":{"@id":"https://lucioduran.com/blog#blog"},"articleSection":"Databases","keywords":"scylladb, cassandra, distributed-databases, low-latency, seastar, cpp, nosql, shard-per-core","wordCount":1367,"timeRequired":"PT8M","workTranslation":[{"@type":"BlogPosting","inLanguage":"es","url":"https://lucioduran.com/es/blog/scylladb-cassandra-compatible-low-latency"}]},{"@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://lucioduran.com/"},{"@type":"ListItem","position":2,"name":"Blog","item":"https://lucioduran.com/blog"},{"@type":"ListItem","position":3,"name":"ScyllaDB: Cassandra-Compatible Low-Latency Database Architecture and Migration","item":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency"}]},{"@type":"FAQPage","mainEntity":[{"@type":"Question","name":"Can I use my existing Cassandra drivers with ScyllaDB?","acceptedAnswer":{"@type":"Answer","text":"Yes. ScyllaDB implements the CQL native protocol, so any Cassandra driver works out of the box. However, ScyllaDB's shard-aware drivers (available for Java, Go, Python, Rust, and C++) route requests directly to the correct shard, cutting cross-shard coordination and reducing p99 latency by 30-60%."}},{"@type":"Question","name":"How does ScyllaDB achieve lower tail latency than Cassandra?","acceptedAnswer":{"@type":"Answer","text":"Three mechanisms: shard-per-core architecture eliminates lock contention, the Seastar framework uses cooperative scheduling with no context switches, and the I/O scheduler provides per-shard fairness. There's no garbage collector, no stop-the-world pauses, no JVM safepoints."}},{"@type":"Question","name":"What's the real-world migration effort from a 12-node Cassandra cluster?","acceptedAnswer":{"@type":"Answer","text":"Typically 3-6 weeks for a production migration including validation. The schema is nearly 1:1 compatible. Main effort goes into tuning compaction strategies (ScyllaDB's Incremental Compaction Strategy has no Cassandra equivalent) and adjusting client-side timeouts downward since ScyllaDB responds faster."}}]}]}],"contentHash":"9ffab5d302be719b5a591714c7ca78e9ea1dd5278e1ae14a24c757c36d0380b3","provenance":{"method":"Deterministic serialization of the server-rendered main element","source":"https://lucioduran.com/blog/scylladb-cassandra-compatible-low-latency","hashAlgorithm":"sha256","hashScope":"JSON document fields before contentHash and provenance","sourceHTMLBytes":65508,"contentBytes":12323}}