Definition
- L1 (Caffeine) – a small, fast, in-process cache with nanosecond lookups; scope is one JVM instance, not shared across instances.
- L2 (Redis) – a shared distributed cache accessed over the network; all instances read and write the same entries.
- Stampede Absorption – when L2 expires, L1 keeps serving per-instance, cutting reloads from "one per request" to "one per instance per L1 TTL window."
- Single-Flight – Caffeine's loading cache ensures only one thread per instance performs the reload; others wait for its result.
- L1 TTL Trade-off – short L1 TTL = instances re-check L2 sooner (less stale) but fall through more often (weaker protection); long L1 TTL = stronger protection but more staleness.
- Broadcast Invalidation – Redis Pub/Sub or a message topic to evict L1 entries across all instances immediately on write.
At National Bank of India the authoritative rate book is in the vault (database), a shared whiteboard in the central hall holds the current rate (Redis, L2), and each teller keeps a sticky note at their window (Caffeine, L1). When the whiteboard gets wiped, each teller still has their sticky note for a little while, so only one runner per window goes to refresh, not one per customer. How long the sticky note stays valid is the trade-off: short note life means fresh figures but more vault trips; long note life means the vault is bothered less but figures can be stale.
Understanding the Concepts
What is the structure of multi-level caching?
The hierarchy is L1 (Caffeine, in-process, nanoseconds) in front of L2 (Redis, shared, sub-millisecond) in front of the database (source of truth, milliseconds). A read flows L1 then L2 then DB, populating upward on return.
package com.bank.cache;
import java.math.BigDecimal;
import java.util.HashMap;
import java.util.Map;
public class MultiLevelCacheDemo {
private final Map<String, BigDecimal> l1 = new HashMap<>(); // Caffeine per-instance
private final Map<String, BigDecimal> l2 = new HashMap<>(); // Redis shared
private final Map<String, BigDecimal> db = Map.of(
"SAV2001", new BigDecimal("75000.00"));
public BigDecimal get(String accountId) {
// Check L1 first (nanoseconds)
BigDecimal val = l1.get(accountId);
if (val != null) { System.out.println("L1 HIT (nanoseconds)"); return val; }
// Check L2 (sub-millisecond network hop)
val = l2.get(accountId);
if (val != null) {
System.out.println("L2 HIT (Redis), populating L1");
l1.put(accountId, val);
return val;
}
// Load from DB (milliseconds)
System.out.println("DB LOAD, populating L2 and L1");
val = db.get(accountId);
l2.put(accountId, val);
l1.put(accountId, val);
return val;
}
public static void main(String[] args) {
MultiLevelCacheDemo cache = new MultiLevelCacheDemo();
cache.get("SAV2001"); // DB LOAD
cache.get("SAV2001"); // L1 HIT
}
}
DB LOAD, populating L2 and L1
L1 HIT (nanoseconds)
How does L1 absorb a stampede when L2 expires?
With only Redis, when a hot key expires, the next read from every instance misses and hits the database simultaneously. With an L1 per instance, each instance may still have the value, so reads continue from L1 without touching Redis or the database.
package com.bank.cache;
public class StampedeAbsorption {
public static void main(String[] args) {
int instances = 20;
int concurrentRequestsPerInstance = 500;
System.out.println("=== Without L1 (Redis only) ===");
int dbHitsNoL1 = instances * concurrentRequestsPerInstance;
System.out.println("Redis key expires → " + dbHitsNoL1 + " concurrent DB reloads");
System.out.println();
System.out.println("=== With L1 (Caffeine + single-flight) ===");
int dbHitsWithL1 = instances; // one per instance, single-flight within each
System.out.println("Redis key expires → each instance serves from L1");
System.out.println("Only " + dbHitsWithL1 + " DB reloads (one per instance)");
System.out.println("Reduction: " + dbHitsNoL1 + " → " + dbHitsWithL1);
}
}
=== Without L1 (Redis only) === Redis key expires → 10000 concurrent DB reloads === With L1 (Caffeine + single-flight) === Redis key expires → each instance serves from L1 Only 20 DB reloads (one per instance) Reduction: 10000 → 20
What is the L1 TTL consistency trade-off?
Because L1 is per-instance and not invalidated when another instance writes, a value in L1 can be stale for up to its TTL even after Redis and the database were updated.
package com.bank.cache;
public class L1TtlTradeoff {
public static void main(String[] args) {
System.out.println("Short L1 TTL (2 seconds):");
System.out.println(" + Instances re-check Redis every 2s → fresh data");
System.out.println(" - Instances fall through to Redis often → weaker stampede protection");
System.out.println();
System.out.println("Long L1 TTL (30 seconds):");
System.out.println(" + Instances serve from L1 for 30s → strong DB protection");
System.out.println(" - A change can take up to 30s to become visible → more staleness");
System.out.println();
System.out.println("Choose based on how much staleness the data tolerates:");
System.out.println(" Displayed balance aggregate → 2-5s L1 TTL is fine");
System.out.println(" Near-real-time rate → very short L1 TTL or no L1");
}
}
Short L1 TTL (2 seconds): + Instances re-check Redis every 2s → fresh data - Instances fall through to Redis often → weaker stampede protection Long L1 TTL (30 seconds): + Instances serve from L1 for 30s → strong DB protection - A change can take up to 30s to become visible → more staleness Choose based on how much staleness the data tolerates: Displayed balance aggregate → 2-5s L1 TTL is fine Near-real-time rate → very short L1 TTL or no L1
How does broadcast invalidation work?
For data requiring prompt cross-instance freshness, publish an eviction message over Redis Pub/Sub so each instance evicts that key from its L1.
package com.bank.cache;
import java.util.Map;
import java.util.HashMap;
public class BroadcastInvalidation {
private final Map<String, String> l1 = new HashMap<>();
public void onInvalidationMessage(String key) {
l1.remove(key);
System.out.println("L1 evicted: " + key + " (broadcast received)");
}
public static void main(String[] args) {
BroadcastInvalidation instance1 = new BroadcastInvalidation();
BroadcastInvalidation instance2 = new BroadcastInvalidation();
instance1.l1.put("SAV2001", "75000");
instance2.l1.put("SAV2001", "75000");
// Writer updates Redis and publishes invalidation
System.out.println("Writer: balance updated in Redis, publishing invalidation");
instance1.onInvalidationMessage("SAV2001");
instance2.onInvalidationMessage("SAV2001");
System.out.println("Both instances now fetch fresh value on next read");
}
}
Writer: balance updated in Redis, publishing invalidation
L1 evicted: SAV2001 (broadcast received)
L1 evicted: SAV2001 (broadcast received)
Both instances now fetch fresh value on next read
How do you compare TTL-based vs broadcast invalidation?
| Strategy | Mechanism | Staleness | Complexity | Best For |
|---|---|---|---|---|
| TTL-based | L1 entries expire after TTL | Up to L1 TTL | Simple | Data tolerating seconds of staleness |
| Broadcast (Pub/Sub) | Writer publishes eviction | Near-immediate | More machinery | Data requiring prompt freshness |
| Combined | Short TTL as backstop + broadcast | Near-immediate | Most complete | Critical shared data |
Why must L1 be bounded?
L1 lives in the JVM heap. Without a size limit, it can consume unbounded memory and cause GC pressure or OOM.
package com.bank.cache;
import com.github.benmanes.caffeine.cache.Caffeine;
import com.github.benmanes.caffeine.cache.Cache;
import java.time.Duration;
public class BoundedL1 {
public static void main(String[] args) {
Cache<String, String> l1 = Caffeine.newBuilder()
.maximumSize(10_000) // bounded by entry count
.expireAfterWrite(Duration.ofSeconds(5)) // bounded by time
.build();
System.out.println("L1 bounded: max 10,000 entries, 5s TTL");
System.out.println("Prevents unbounded heap growth per instance");
}
}
L1 bounded: max 10,000 entries, 5s TTL
Prevents unbounded heap growth per instance
What are common mistakes?
Using an unbounded in-process cache (a plain HashMap) as L1 without size limits or TTL, causing memory leaks and eventual OOM under load.
package com.bank.cache;
import java.util.HashMap;
// WRONG: unbounded HashMap as L1
public class WrongL1 {
static HashMap<String, String> l1 = new HashMap<>();
// Grows forever, no eviction, no TTL → OOM under load
}
// CORRECT: Caffeine with size and time bounds
// Caffeine.newBuilder().maximumSize(10_000).expireAfterWrite(5, SECONDS).build();
What are common interview follow-up questions?
- How do you choose the L1 TTL for a payment dashboard?
- What happens if the Redis Pub/Sub message is lost?
- How do you measure L1 hit rate to justify the extra layer?
- Can L1 use refresh-ahead while L2 uses cache-aside?
Visual Overview
Multi-Level Cache Stampede Absorption
─────────────────────────────────────
Without L1 (Redis only):
┌──────────┐ ┌──────────┐ ┌──────────┐
│Instance 1│ │Instance 2│ │Instance 3│
└────┬─────┘ └────┬─────┘ └────┬─────┘
│ │ │
└────────────┼────────────┘
▼
┌──────────────────┐
│ Redis (EXPIRED) │
└────────┬─────────┘
▼
┌──────────────────┐
│ Database │ ← 10,000 concurrent hits!
└──────────────────┘
With L1 (Caffeine + Redis):
┌──────────┐ ┌──────────┐ ┌──────────┐
│Instance 1│ │Instance 2│ │Instance 3│
│ L1: HIT │ │ L1: HIT │ │ L1: HIT │
└──────────┘ └──────────┘ └──────────┘
│ │ │
│ (only 1 req per instance falls through)
└────────────┼────────────┘
▼
┌──────────────────┐
│ Database │ ← only 3 hits!
└──────────────────┘
Key Takeaway
Multi-level caching layers per-process Caffeine (L1) in front of shared Redis (L2) in front of the database. The key benefit is stampede absorption: when a Redis key expires, each instance still has the value in L1, so reads continue from L1 without touching Redis or the database. Only one request per instance falls through to reload, cutting database hits from thousands to a handful.
The consistency trade-off is governed by the L1 TTL. A short L1 TTL means instances re-check Redis sooner for less staleness, but fall through more often for weaker protection. A long L1 TTL gives strong database protection but more staleness. For prompt cross-instance freshness, add broadcast invalidation over Redis Pub/Sub.
L1 must be bounded with a maximum size and eviction policy to prevent unbounded heap growth. The layers can use different patterns: L1 as refresh-ahead to never block, L2 as cache-aside with single-flight. A burst is progressively filtered: L1 absorbs most per instance, L2 absorbs most of what gets through, and the database sees only a trickle.
Remember one sentence: layer a bounded Caffeine L1 in front of shared Redis L2 so that when L2 expires, each instance serves from L1 and the database sees one reload per instance, not one per request.
---