PACELC Theorem: Extending CAP to the Healthy-System Case

• CAP blind spot – CAP only describes behavior during a network partition, which is the rare failure mode.
• PACELC extension – if there is a Partition, choose Availability or Consistency; Else (normally), choose Latency or Consistency.
• Two-letter label – a system is classified as PA/EL, PC/EC, etc., describing behavior in both regimes.
• Everyday trade-off – even with a healthy network, writes must decide how many replica acks to wait for before responding.
• Per-operation tuning – balance writes can be EC while feed reads can be EL, tuned per query.

What is the CAP theorem and what does it miss?

CAP states that during a network partition, a distributed system must choose between Consistency and Availability. But a well-run system is partitioned a tiny fraction of the time.

CAP covers:     Partition happens ──▶ Choose A or C
CAP ignores:    System is healthy  ──▶ ???

PACELC fills:   Partition happens  ──▶ Choose A or C
                Else (healthy)     ──▶ Choose L or C

What is the PACELC theorem?

Proposed by Daniel Abadi, PACELC says every distributed system makes two trade-off decisions.

PACELC = PAC + ELC

If Partition:
  Choose Availability (serve on both sides, risk stale reads)
  Or Consistency (refuse/block to avoid divergence)

Else (no partition):
  Choose Latency (ack after one replica, fast but possibly stale)
  Or Consistency (wait for all replicas, correct but slower)

Why does the Else case matter?

Partitions are rare, but every single write faces the "wait for all acks or return after one" question.

package com.bank.service;

import java.math.BigDecimal;

public class BalanceUpdateService {

    // EC approach: wait for replica ack (consistent, higher latency)
    public void updateBalanceEC(long accountId, BigDecimal newBalance) {
        // Write to primary
        // Wait for at least one replica to acknowledge
        // THEN return success to the client
        // Read after write is guaranteed consistent
    }

    // EL approach: return after primary ack (fast, possibly stale)
    public void updateBalanceEL(long accountId, BigDecimal newBalance) {
        // Write to primary
        // Return success immediately
        // Replicate asynchronously in the background
        // A read on a lagging replica may see the old balance
    }
}

How are common databases classified?

| System | During Partition (P) | Else (E) | Label |
|---|---|---|---|
| PostgreSQL (sync repl) | PC (refuse if replica down) | EC (wait for replica ack) | PC/EC |
| DynamoDB (default) | PA (serve on both sides) | EL (fast, eventually consistent) | PA/EL |
| Cassandra (tunable) | PA (default) | EL (ONE) or EC (QUORUM) | PA/EL or PA/EC |
| MongoDB (majority write) | PC | EC | PC/EC |
| MongoDB (w:1) | PA | EL | PA/EL |

How does tunable consistency work?

Cassandra and DynamoDB let you choose per request.

package com.bank.repository;

public class CassandraExample {

    // Tunable per query:
    // ConsistencyLevel.ONE  = EL (fast, possibly stale)
    // ConsistencyLevel.QUORUM = EC (consistent, higher latency)
    // ConsistencyLevel.ALL  = strongest EC (slowest)

    public void writeBalance() {
        // Balance writes: QUORUM (EC) for correctness
        // session.execute(query.setConsistencyLevel(ConsistencyLevel.QUORUM));
    }

    public void readFeed() {
        // Feed reads: ONE (EL) for speed, staleness is harmless
        // session.execute(query.setConsistencyLevel(ConsistencyLevel.ONE));
    }
}

What is the real-world impact of choosing EL for money?

If balance writes use EL (async replication), a read right after a write might hit a lagging replica and show the old balance.

Timeline with EL (async replication):
  T1: Client writes balance = 50000 to primary
  T2: Primary acks immediately (fast!)
  T3: Client reads balance from replica
  T4: Replica has not received the write yet
  T5: Client sees balance = 45000 (stale!)

For money, this is unacceptable. Use EC (synchronous) for financial data.

How does PostgreSQL implement PC/EC?

-- PostgreSQL: synchronous replication for EC
-- postgresql.conf on primary:
-- synchronous_standby_names = 'replica1'
-- synchronous_commit = on

-- Behavior:
-- Every COMMIT waits for replica1 to confirm WAL flush
-- If replica1 is down (partition), the primary blocks: PC
-- During normal operation, latency includes replica round-trip: EC

How do you apply PACELC per operation?

Pankaj classifies operations at National Bank of India by tolerance for staleness.

┌─────────────────────┬────────────┬─────────────────────────────┐
│ Operation            │ PACELC     │ Reason                       │
├─────────────────────┼────────────┼─────────────────────────────┤
│ Balance update        │ EC         │ Money must be correct         │
│ Balance read          │ EC         │ Must see latest balance       │
│ Transaction feed      │ EL         │ Brief staleness is harmless   │
│ Notification counter  │ EL         │ Approximate is fine           │
│ Audit log write       │ EC         │ Must be durable and ordered   │
└─────────────────────┴────────────┴─────────────────────────────┘

What should a senior engineer know?

• PACELC should be applied per operation, not globally.
• The EL-versus-EC decision dominates daily p99 latency and read-your-writes behavior.
• CAP-only thinking fixates on the rare partition case and ignores the everyday trade-off.
• Many databases expose the choice as tunable consistency levels or write concerns.
• Always test read-after-write behavior under the chosen consistency level.