$ man cap # Brewer, 2000 · Gilbert & Lynch, 2002

CAP: what breaks when the network breaks

You keep copies of the same data on several machines (replicas) so the system survives a dead server. CAP asks a single, sharp question:

When the replicas can’t talk to each other, and a request arrives, do you answer anyway or refuse?

That’s the whole theorem. Everything else is detail.

C · Consistency

Everyone sees the latest write

After a write succeeds, every read from any node returns that value (linearizability).

A · Availability

Every live node answers

Any request to a non-failed node gets a real response: not an error, not a timeout.

P · Partition

The network splits

Messages between nodes are lost or delayed. On real networks this will happen.

See it happen

Pick a mode, then step through. Same cluster, same failure: only the choice at step 4 differs.

$ ./cluster --replicas 3 --partition node3
Three replicas, one cut cable
During a partition, Node 3 should:
✂ ✂ CLIENT NODE 1x = 1ok NODE 2x = 1ok NODE 3x = 1ok
NO.STEPFROMTOTYPEINFO
101clientnode1WRITEx = 2 → ok
201node1node2, node3REPLICATEx = 2 → ack ×2
302node1, node2node3LINK DOWNnetwork partition
403clientnode1WRITEx = 3 → ok (quorum: node1 + node2)
503node1node3REPLICATEx = 3 → lost
604clientnode3READx ?
704node3clientERROR503 · can't reach quorum (consistent, not available)
704node3clientRESPONSEx = 2 · stale (available, not consistent)
805node1node3SYNCx = 3 → replicas converge
  1. Three replicas hold the same value x = 1. Press Play, or Step through it at your own pace (← → also work).
  2. A client writes x = 2. Node 1 copies it to the others. No partition → consistent and available. Easy.
  3. The network splits. Node 3 is alive, it just can't hear Node 1 or Node 2.
  4. A new write x = 3 lands on Node 1. It reaches Node 2, but the copy to Node 3 is lost.
  5. A client asks Node 3 for x. CP: it refuses: correct but unavailable. AP: it answers with old data: available but inconsistent. Toggle the mode to compare.
  6. The network heals and Node 3 catches up to x = 3. In AP this is eventual consistency: replicas converge once messages flow again.

So the honest reading of CAP is: P is not optional. Networks fail, so during a partition you choose between C and A. “CA” only makes sense on a single machine, where there is no network to split.

Real systems, real choices

System During a partition Why
PostgreSQL (sync replica), etcd, ZooKeeper, Spanner CP Wrong answers are worse than no answer (money, locks, leader election).
Cassandra, DynamoDB, Riak (default settings) AP Shopping carts, feeds, metrics: an old answer beats an error page.
Cassandra / DynamoDB with QUORUM or strong reads closer to CP The choice is often per query, not per database.
Kafka tunable acks=all + min.insync.replicas trades availability for durability.

Beyond CAP: PACELC

CAP only talks about the rare moment of a partition. Most of the time the network is fine, and you still pay for consistency in latency: a write must wait for replicas to confirm.

PACELC: if Partition → choose A or C; Else → choose Latency or Consistency.

Cassandra is PA/EL (fast, eventually consistent). Spanner is PC/EC (consistent, pays with latency and atomic clocks). See PACELC and Consistency vs Eventual Consistency.

Common misconceptions

  • “Pick any 2 of 3.” You can’t drop P. The real choice is C or A while partitioned.
  • “AP means always inconsistent.” No: replicas converge once the partition heals.
  • CAP’s C is not ACID’s C. CAP’s C is about replicas agreeing; ACID’s C is about invariants inside one database. See ACID.

Learning Resources

Articles

Videos

Books

  • Designing Data-Intensive Applications (2nd ed.), Martin Kleppmann and Chris Riccomini: chapters on replication and consistency