CAP: what breaks when the network breaks
You keep copies of the same data on several machines (replicas) so the system survives a dead server. CAP asks a single, sharp question:
When the replicas can’t talk to each other, and a request arrives, do you answer anyway or refuse?
That’s the whole theorem. Everything else is detail.
Everyone sees the latest write
After a write succeeds, every read from any node returns that value (linearizability).
Every live node answers
Any request to a non-failed node gets a real response: not an error, not a timeout.
The network splits
Messages between nodes are lost or delayed. On real networks this will happen.
See it happen
Pick a mode, then step through. Same cluster, same failure: only the choice at step 4 differs.
| NO. | STEP | FROM | TO | TYPE | INFO |
|---|---|---|---|---|---|
| 1 | 01 | client | node1 | WRITE | x = 2 → ok |
| 2 | 01 | node1 | node2, node3 | REPLICATE | x = 2 → ack ×2 |
| 3 | 02 | node1, node2 | node3 | LINK DOWN | network partition |
| 4 | 03 | client | node1 | WRITE | x = 3 → ok (quorum: node1 + node2) |
| 5 | 03 | node1 | node3 | REPLICATE | x = 3 → lost |
| 6 | 04 | client | node3 | READ | x ? |
| 7 | 04 | node3 | client | ERROR | 503 · can't reach quorum (consistent, not available) |
| 7 | 04 | node3 | client | RESPONSE | x = 2 · stale (available, not consistent) |
| 8 | 05 | node1 | node3 | SYNC | x = 3 → replicas converge |
- Three replicas hold the same value x = 1. Press Play, or Step through it at your own pace (← → also work).
- A client writes x = 2. Node 1 copies it to the others. No partition → consistent and available. Easy.
- The network splits. Node 3 is alive, it just can't hear Node 1 or Node 2.
- A new write x = 3 lands on Node 1. It reaches Node 2, but the copy to Node 3 is lost.
- A client asks Node 3 for x. CP: it refuses: correct but unavailable. AP: it answers with old data: available but inconsistent. Toggle the mode to compare.
- The network heals and Node 3 catches up to x = 3. In AP this is eventual consistency: replicas converge once messages flow again.
So the honest reading of CAP is: P is not optional. Networks fail, so during a partition you choose between C and A. “CA” only makes sense on a single machine, where there is no network to split.
Real systems, real choices
| System | During a partition | Why |
|---|---|---|
| PostgreSQL (sync replica), etcd, ZooKeeper, Spanner | CP | Wrong answers are worse than no answer (money, locks, leader election). |
| Cassandra, DynamoDB, Riak (default settings) | AP | Shopping carts, feeds, metrics: an old answer beats an error page. |
Cassandra / DynamoDB with QUORUM or strong reads |
closer to CP | The choice is often per query, not per database. |
| Kafka | tunable | acks=all + min.insync.replicas trades availability for durability. |
Beyond CAP: PACELC
CAP only talks about the rare moment of a partition. Most of the time the network is fine, and you still pay for consistency in latency: a write must wait for replicas to confirm.
PACELC: if Partition → choose A or C; Else → choose Latency or Consistency.
Cassandra is PA/EL (fast, eventually consistent). Spanner is PC/EC (consistent, pays with latency and atomic clocks). See PACELC and Consistency vs Eventual Consistency.
Common misconceptions
- “Pick any 2 of 3.” You can’t drop P. The real choice is C or A while partitioned.
- “AP means always inconsistent.” No: replicas converge once the partition heals.
- CAP’s C is not ACID’s C. CAP’s C is about replicas agreeing; ACID’s C is about invariants inside one database. See ACID.
Learning Resources
Articles
- A critique of the CAP theorem: Martin Kleppmann
- CAP Twelve Years Later: Eric Brewer
- Jepsen analyses: how real databases behave under partitions
Videos
Books
- Designing Data-Intensive Applications (2nd ed.), Martin Kleppmann and Chris Riccomini: chapters on replication and consistency