Definition
- LRU -- evicts the entry untouched for the longest time; suits temporal locality.
- LFU -- evicts the entry with the fewest accesses; suits skewed popularity.
- FIFO -- evicts the oldest-inserted entry; cheap but usually a poor cache policy.
- TTL -- removes entries after a max age for freshness; composes with capacity policies.
- Random -- picks an arbitrary victim; surprisingly competitive under heavy skew.
- Session mismatch -- evicting an active session logs the user out; size for expected sessions, not capacity eviction.
- Hit-rate economics -- on a miss you pay the Redis hop and the DB query; below roughly 50% hit rate the cache is overhead.
Rajesh keeps a small desk tray of ten customer files at National Bank of India. When the tray is full, he must remove one. LRU removes the one untouched longest. LFU removes the least consulted. FIFO removes whichever arrived first. If he removes a file he is about to need, he makes two trips to the vault instead of one, and his tray has made him slower, not faster.
Understanding the Concepts
What are the main eviction policies?
Each policy answers one question: when the cache is full, which entry leaves?
| Policy | Evicts | Best For | Weakness |
|---|---|---|---|
| LRU | Least recently used | Temporal locality (feeds, balances) | Scan pollution |
| LFU | Least frequently used | Skewed popularity (top symbols) | Aging problem |
| FIFO | Oldest inserted | Simple buffers | Ignores usage |
| TTL | Entries past max age | Freshness (displayed balance) | Not capacity control |
| Random | Arbitrary | Cheap bookkeeping | Not targeted |
package com.bank.cache;
public class PolicyOverview {
public static void main(String[] args) {
System.out.println("LRU: evicts least recently used");
System.out.println("LFU: evicts least frequently used");
System.out.println("FIFO: evicts oldest inserted");
System.out.println("TTL: evicts after max age");
System.out.println("Random: evicts arbitrary entry");
}
}
Expected output:
LRU: evicts least recently used
LFU: evicts least frequently used
FIFO: evicts oldest inserted
TTL: evicts after max age
Random: evicts arbitrary entry
How do you configure LRU in Caffeine?
Caffeine is a high-performance local cache for Java. Setting maximumSize enables size-based eviction (LRU-like behavior with Window-TinyLFU), and expireAfterWrite adds TTL. The recordStats() call enables hit-rate monitoring.
package com.bank.cache;
import com.github.benmanes.caffeine.cache.Caffeine;
import com.github.benmanes.caffeine.cache.Cache;
import java.math.BigDecimal;
import java.time.Duration;
public class BalanceLocalCache {
private final Cache<String, BigDecimal> cache = Caffeine.newBuilder()
.maximumSize(10_000)
.expireAfterWrite(Duration.ofSeconds(30))
.recordStats()
.build();
public BigDecimal get(String accountId) {
return cache.getIfPresent(accountId);
}
public void put(String accountId, BigDecimal balance) {
cache.put(accountId, balance);
}
public double hitRate() {
return cache.stats().hitRate(); // monitor this with Prometheus
}
public static void main(String[] args) {
BalanceLocalCache c = new BalanceLocalCache();
c.put("SAV2001", new BigDecimal("150000.00"));
System.out.println("Cached balance: " + c.get("SAV2001"));
System.out.println("Hit rate: " + c.hitRate());
}
}
Expected output:
Cached balance: 150000.00
Hit rate: 1.0
How is LRU different from LFU?
LRU tracks when each entry was last accessed. LFU tracks how many times each entry was accessed. Under scan pollution (a batch job touching thousands of keys once), LRU evicts the hot entries to make room for one-hit wonders. LFU resists this because one-hit entries have low frequency.
| Aspect | LRU | LFU |
|---|---|---|
| Tracks | Last access time | Access count |
| Evicts | Untouched longest | Fewest accesses |
| Strength | Adapts quickly to shifting popularity | Resists scan pollution |
| Weakness | Scan pollution evicts hot entries | Aging: old popular items never leave |
| Use case | Content feeds, recent transactions | Top trading symbols, hot products |
package com.bank.cache;
public class LruVsLfu {
public static void main(String[] args) {
// Scenario: batch report scans 10,000 accounts
// LRU: all 10,000 enter cache, hot balances evicted
System.out.println("LRU after scan: hot balances EVICTED, cache full of cold data");
// LFU: one-hit-wonder entries have frequency 1, hot balances have frequency 1000+
System.out.println("LFU after scan: hot balances RETAINED, cold scan entries evicted");
}
}
Expected output:
LRU after scan: hot balances EVICTED, cache full of cold data
LFU after scan: hot balances RETAINED, cold scan entries evicted
Why is a session store a mismatch for capacity eviction?
A session is active data. Evicting it under memory pressure logs a working user out mid-transfer. A burst of new logins can push Rajesh's active session out of the cache, ending his session while he is transferring money to Priya.
package com.bank.cache;
import com.github.benmanes.caffeine.cache.Caffeine;
import com.github.benmanes.caffeine.cache.Cache;
import java.time.Duration;
public class SessionStoreConfig {
// WRONG: LRU on session store
// A burst of new logins evicts Rajesh's active session
// Cache<String, Session> wrong = Caffeine.newBuilder()
// .maximumSize(1000).build(); // capacity eviction kills sessions
// CORRECT: size for expected concurrent sessions + TTL matching session timeout
public static Cache<String, String> correct(int expectedConcurrent) {
return Caffeine.newBuilder()
.maximumSize(expectedConcurrent) // sized to NEVER evict active sessions
.expireAfterAccess(Duration.ofMinutes(30)) // TTL = session timeout
.build();
}
public static void main(String[] args) {
Cache<String, String> sessions = correct(50_000);
sessions.put("session:rajesh", "SAV2001");
System.out.println("Session cached: " + sessions.getIfPresent("session:rajesh"));
System.out.println("Sized for 50,000 concurrent; TTL 30 min; no capacity eviction");
}
}
Expected output:
Session cached: SAV2001
Sized for 50,000 concurrent; TTL 30 min; no capacity eviction
Why does a low hit rate make a cache a net negative?
On a hit, the client pays only the Redis hop (0.5 ms). On a miss, the client pays the Redis hop (0.5 ms) plus the DB query (5 ms). With a 20% hit rate, 80% of requests pay 5.5 ms and only 20% save time. The average is worse than going to the database directly, plus you pay the cost of running Redis.
package com.bank.cache;
public class HitRateEconomics {
public static void main(String[] args) {
double redisHopMs = 0.5;
double dbQueryMs = 5.0;
double hitRate = 0.20;
double hitCost = redisHopMs;
double missCost = redisHopMs + dbQueryMs;
double avgWithCache = hitRate * hitCost + (1 - hitRate) * missCost;
double avgWithoutCache = dbQueryMs;
System.out.printf("With cache (%.0f%% hit rate): %.1f ms avg%n",
hitRate * 100, avgWithCache);
System.out.printf("Without cache: %.1f ms avg%n", avgWithoutCache);
System.out.println(avgWithCache < avgWithoutCache
? "Cache is HELPFUL" : "Cache is OVERHEAD -- net negative!");
}
}
Expected output:
With cache (20% hit rate): 4.5 ms avg
Without cache: 5.0 ms avg
Cache is OVERHEAD -- net negative!
What causes low hit rates?
Three common causes destroy cache effectiveness:
- Wrong eviction policy: scan pollution evicts hot entries.
- Cache sized too small for the working set.
- Access pattern with no locality: random reads across millions of keys.
package com.bank.cache;
public class LowHitRateCauses {
public static void main(String[] args) {
// Cause 1: scan pollution
System.out.println("Batch report scans 10K accounts --> LRU evicts hot balances");
// Cause 2: undersized cache
long workingSet = 100_000;
long cacheSize = 1_000; // 1% of working set
System.out.println("Cache holds " + cacheSize + " of " + workingSet
+ " items --> thrashing");
// Cause 3: no locality
System.out.println("Random reads across 50M accounts --> every read is a miss");
}
}
Expected output:
Batch report scans 10K accounts --> LRU evicts hot balances
Cache holds 1000 of 100000 items --> thrashing
Random reads across 50M accounts --> every read is a miss
What are common mistakes?
The most common mistake is treating the cache policy as a detail and never monitoring the hit rate.
package com.bank.cache;
public class CacheMistakes {
public static void main(String[] args) {
// WRONG: deploying a cache and never monitoring hit rate
System.out.println("WRONG: cache deployed, no hit-rate metric, no alert");
System.out.println("Result: 15% hit rate for 6 months, nobody noticed");
// CORRECT: Prometheus gauge on hit rate, alert below 50%
System.out.println("CORRECT: Prometheus gauge on cache_hit_rate");
System.out.println("Alert fires when hit rate drops below 50%");
}
}
Expected output:
WRONG: cache deployed, no hit-rate metric, no alert
Result: 15% hit rate for 6 months, nobody noticed
CORRECT: Prometheus gauge on cache_hit_rate
Alert fires when hit rate drops below 50%
What should a senior engineer know?
- TTL composes with capacity eviction: use LRU for "who leaves when full" and TTL for "when any entry is too stale."
- Caffeine's Window-TinyLFU combines recency with an admission filter to resist one-hit-wonder pollution.
- Put a Prometheus gauge on hit rate and alert if it falls below 50%.
- The eviction policy is not a detail; the wrong policy quietly turns your cache into overhead.
package com.bank.cache;
public class SeniorInsights {
public static void main(String[] args) {
System.out.println("1. TTL + LRU compose: TTL for freshness, LRU for capacity");
System.out.println("2. Window-TinyLFU resists scan pollution automatically");
System.out.println("3. Monitor hit rate; alert below 50%");
System.out.println("4. Wrong policy = silent overhead, not a loud error");
}
}
Expected output:
1. TTL + LRU compose: TTL for freshness, LRU for capacity
2. Window-TinyLFU resists scan pollution automatically
3. Monitor hit rate; alert below 50%
4. Wrong policy = silent overhead, not a loud error
Visual Overview
Cache Hit-Rate Economics HIT (20% of requests): Client ──► Redis ──► return = 0.5 ms MISS (80% of requests): Client ──► Redis ──► (nothing) ──► DB ──► cache + return = 5.5 ms Average with 20% hit rate: 4.5 ms Average without cache: 5.0 ms Cache is OVERHEAD (0.5 ms slower + Redis cost!) Average with 90% hit rate: 1.0 ms Cache earns its place Session Store Mismatch ┌─────────────────────────────────────────────┐ │ Burst of new logins fills cache │ │ LRU evicts Rajesh's ACTIVE session │ │ Rajesh appears logged out mid-transfer! │ │ Fix: size for expected sessions + TTL only │ └─────────────────────────────────────────────┘ Eviction Policy Decision ┌──────────────────────────────────────┐ │ Temporal locality? --> LRU │ │ Skewed popularity? --> LFU │ │ Session data? --> TTL only │ │ Simple buffer? --> FIFO │ │ Always monitor hit rate! │ └──────────────────────────────────────┘
Key Takeaway
Cache eviction policies decide which entry leaves when the cache is full. LRU suits workloads with temporal locality like content feeds and hot balances. LFU suits skewed popularity like top trading symbols. TTL is about freshness, not capacity, and composes on top of capacity policies. FIFO is rarely a good cache policy because it ignores usage entirely.
The session store is the classic mismatch: a session is active data, and evicting it logs a user out mid-transfer. The fix is to size the store for expected concurrent sessions and let entries leave only by TTL matching the session timeout.
At National Bank of India, a balance cache was being polluted by a batch report scanning thousands of accounts, dropping the hit rate to near zero. Pankaj fixed it with Caffeine's frequency-aware admission filter and a Prometheus alert on hit rate. A cache at 20% hit rate is actively harmful: on 80% of requests you add latency for zero benefit.
Remember one sentence: match the eviction policy to the access pattern, size session stores for expected sessions with TTL only, and monitor the hit rate because a cache below 50% hit rate is overhead, not optimization.
---