Definition
- Refresh-Ahead – a proactive caching strategy that reloads a popular value in the background before its TTL expires, so readers always hit a fresh cached value and never experience a blocking miss.
- refreshAfterWrite (Caffeine) – returns the existing value immediately and triggers an async background reload, unlike expireAfterWrite which blocks on reload.
- Cache Stampede (Thundering Herd) – many keys expire at the same instant and all readers simultaneously trigger expensive reloads, overwhelming the database.
- Probabilistic Early Recompute (XFetch) – each read refreshes with a probability that rises as expiry nears, scattering refreshes over a window instead of concentrating them at the deadline.
- TTL Jitter – adding a small random offset to each key's TTL so a batch of keys does not expire together.
- Single-Flight – ensuring only one thread actually performs the reload per key, so concurrent readers share the result.
At National Bank of India the lobby displays today's gold rate computed from a slow external feed, valid for five minutes. With plain expiry, at exactly five minutes the display blanks and the next customer forces the clerk to run the slow computation while everyone waits. Refresh-ahead is the clerk who quietly recomputes in the back office a few seconds early so the screen never goes blank. Probabilistic early recompute is the clever twist: each screen rolls a dice that becomes more likely to say "refresh now" the closer it gets to expiry, so refreshes trickle in spread across the last minute.
Understanding the Concepts
What is refresh-ahead caching?
Refresh-ahead exists because plain expiry has a latency cost on every expiry. When a value's TTL elapses, the next reader gets a miss and must wait for the full reload. Refresh-ahead removes that spike by reloading proactively in the background before expiry, so the cache always has a fresh value ready.
package com.bank.cache;
import com.github.benmanes.caffeine.cache.Caffeine;
import com.github.benmanes.caffeine.cache.LoadingCache;
import java.math.BigDecimal;
import java.time.Duration;
public class RefreshAheadExample {
private final LoadingCache<String, BigDecimal> rateCache = Caffeine.newBuilder()
.maximumSize(1000)
.refreshAfterWrite(Duration.ofMinutes(1)) // refresh proactively
.expireAfterWrite(Duration.ofMinutes(5)) // hard expiry as backstop
.build(this::fetchGoldRate);
private BigDecimal fetchGoldRate(String rateKey) {
System.out.println("Background refresh: fetching gold rate from external feed");
return new BigDecimal("62500.00"); // price per 10g
}
public BigDecimal getRate(String rateKey) {
return rateCache.get(rateKey); // never blocks, returns stale value during refresh
}
public static void main(String[] args) {
RefreshAheadExample svc = new RefreshAheadExample();
System.out.println("Gold rate: " + svc.getRate("GOLD_10G_INR"));
System.out.println("Gold rate: " + svc.getRate("GOLD_10G_INR")); // instant, no reload
}
}
Background refresh: fetching gold rate from external feed
Gold rate: 62500.00
Gold rate: 62500.00
How is refreshAfterWrite different from expireAfterWrite?
With expireAfterWrite, the entry is removed after the duration and the next access blocks on a reload. With refreshAfterWrite, the entry is eligible for an async refresh, but the first read returns the existing value immediately.
| Behavior | expireAfterWrite | refreshAfterWrite |
|---|---|---|
| After duration | Entry removed | Entry eligible for refresh |
| Next read | Blocks on reload | Returns stale value instantly |
| Reload | Synchronous, reader waits | Asynchronous, background thread |
| Reader latency | Spike on every expiry | Always fast |
package com.bank.cache;
public class ExpireVsRefresh {
public static void main(String[] args) {
System.out.println("expireAfterWrite(5min): entry dies at 5min, next reader BLOCKS");
System.out.println("refreshAfterWrite(1min) + expireAfterWrite(5min):");
System.out.println(" At 1min: next reader gets stale value, background refresh starts");
System.out.println(" At 5min: hard expiry removes truly abandoned keys");
System.out.println(" Hot keys are ALWAYS fresh, no reader ever blocks");
}
}
expireAfterWrite(5min): entry dies at 5min, next reader BLOCKS
refreshAfterWrite(1min) + expireAfterWrite(5min):
At 1min: next reader gets stale value, background refresh starts
At 5min: hard expiry removes truly abandoned keys
Hot keys are ALWAYS fresh, no reader ever blocks
What is a cache stampede (thundering herd)?
If many keys share the same expiry time, they all expire at the same instant and every reader simultaneously triggers a reload. The database is suddenly hit by a synchronized flood.
package com.bank.cache;
public class CacheStampedeDemo {
public static void main(String[] args) {
int keysLoadedTogether = 10_000;
int ttlSeconds = 300;
System.out.println(keysLoadedTogether + " keys all cached at the same second");
System.out.println("All expire at second " + ttlSeconds + " simultaneously");
System.out.println("At second " + ttlSeconds + ": " + keysLoadedTogether
+ " concurrent DB reloads hit at once = STAMPEDE");
System.out.println("Database overwhelmed, latency spikes, possible outage");
}
}
10000 keys all cached at the same second
All expire at second 300 simultaneously
At second 300: 10000 concurrent DB reloads hit at once = STAMPEDE
Database overwhelmed, latency spikes, possible outage
How does probabilistic early recompute (XFetch) work?
On each read, you compute how close the value is to expiry and refresh early with a probability that increases as expiry approaches. Refreshes scatter over a window instead of concentrating at the deadline.
package com.bank.cache;
import java.util.concurrent.ThreadLocalRandom;
public class ProbabilisticEarlyRecompute {
static boolean shouldRefreshEarly(long computeTimeMs, long timeToExpiryMs, double beta) {
// XFetch: probability grows as expiry nears
double random = -Math.log(ThreadLocalRandom.current().nextDouble());
double threshold = computeTimeMs * beta * random;
return threshold >= timeToExpiryMs;
}
public static void main(String[] args) {
long computeTime = 200; // ms to compute the value
double beta = 1.0;
// Far from expiry: low chance of early refresh
int earlyAt60s = 0;
for (int i = 0; i < 1000; i++)
if (shouldRefreshEarly(computeTime, 60_000, beta)) earlyAt60s++;
System.out.println("60s to expiry: " + earlyAt60s + "/1000 would refresh early");
// Close to expiry: high chance
int earlyAt1s = 0;
for (int i = 0; i < 1000; i++)
if (shouldRefreshEarly(computeTime, 1_000, beta)) earlyAt1s++;
System.out.println("1s to expiry: " + earlyAt1s + "/1000 would refresh early");
}
}
60s to expiry: 3/1000 would refresh early
1s to expiry: 187/1000 would refresh early
How does TTL jitter prevent batch expiry?
The cheapest defense against stampedes: add random jitter to each key's TTL so a batch loaded together gets slightly different expiry times.
package com.bank.cache;
import java.util.concurrent.ThreadLocalRandom;
public class TtlJitter {
static long jitteredTtl(long baseTtlSeconds, long jitterRangeSeconds) {
return baseTtlSeconds + ThreadLocalRandom.current().nextLong(jitterRangeSeconds);
}
public static void main(String[] args) {
long baseTtl = 300; // 5 minutes
long jitter = 60; // up to 1 minute of jitter
System.out.println("Base TTL: " + baseTtl + "s");
for (int i = 0; i < 5; i++) {
System.out.println("Key-" + i + " TTL: " + jitteredTtl(baseTtl, jitter) + "s");
}
System.out.println("Keys expire at different times, no synchronized stampede");
}
}
Base TTL: 300s
Key-0 TTL: 342s
Key-1 TTL: 317s
Key-2 TTL: 355s
Key-3 TTL: 308s
Key-4 TTL: 329s
Keys expire at different times, no synchronized stampede
What is single-flight (one-loader)?
Single-flight ensures only one thread actually performs the reload for a given key. Other threads either wait briefly or return the stale value.
package com.bank.cache;
import java.math.BigDecimal;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.CompletableFuture;
public class SingleFlight {
private final ConcurrentHashMap<String, CompletableFuture<BigDecimal>> inFlight =
new ConcurrentHashMap<>();
public BigDecimal load(String key) {
CompletableFuture<BigDecimal> future = inFlight.computeIfAbsent(key, k -> {
System.out.println("Only ONE thread loads " + k);
return CompletableFuture.supplyAsync(() -> new BigDecimal("62500.00"));
});
BigDecimal result = future.join();
inFlight.remove(key);
return result;
}
public static void main(String[] args) {
SingleFlight sf = new SingleFlight();
// Even if 100 threads call load("GOLD"), only one actually fetches
System.out.println("Rate: " + sf.load("GOLD_10G_INR"));
}
}
Only ONE thread loads GOLD_10G_INR
Rate: 62500.00
What are common mistakes?
Using refresh-ahead for data that must never be stale, like an authoritative balance used to authorize a debit. Refresh-ahead deliberately trades freshness for zero-latency reads.
package com.bank.cache;
// WRONG: refresh-ahead on authoritative debit balance
// Reader gets stale value, authorizes debit on outdated balance
// CORRECT: refresh-ahead on displayed aggregate (dashboard, rate screen)
// A few seconds of staleness is fine, and no reader ever blocks
What are common interview follow-up questions?
- How do you choose the refresh interval relative to the hard expiry?
- What happens for a key that is never read: does refresh-ahead waste resources?
- How does Caffeine's AsyncLoadingCache combine refresh-ahead with single-flight?
- When would you choose time-based invalidation versus event-based invalidation?
Visual Overview
Plain Expiry vs Refresh-Ahead
─────────────────────────────
Plain Expiry:
┌─────────┐ TTL expires ┌──────────┐
│ Cached │────────────────────►│ EXPIRED │
│ (fast) │ │ next read│
└─────────┘ │ BLOCKS │
└──────────┘
Refresh-Ahead:
┌─────────┐ near TTL, bg refresh ┌──────────┐
│ Cached │────────────────────────►│ Refreshed│
│ (fast) │ reader gets stale val │ (fresh) │
└─────────┘ instantly, no block └──────────┘
Probabilistic Early Recompute
─────────────────────────────
Time ──────────────────────────────────► Expiry
│ │
│ P(refresh) ≈ 0 P(refresh) grows │
│ ····· ····· //// █████████ │
│ │
Refreshes scattered Not all at once
Key Takeaway
Refresh-ahead caching proactively reloads a popular value in the background before its TTL expires, so readers always get a fast hit and never experience a blocking miss. Caffeine implements this with refreshAfterWrite, which returns the stale value immediately while one background thread reloads, unlike expireAfterWrite which blocks the reader.
The danger is the cache stampede: if many keys share the same expiry, they all expire at once and a flood of readers simultaneously hammers the database. Probabilistic early recompute (XFetch) solves this by making each read refresh with a probability that rises as expiry nears, scattering refreshes over a window. Per-key TTL jitter is the cheapest complement, and single-flight ensures only one thread actually reloads per key.
Refresh-ahead trades a little freshness for zero latency, which is the right trade for a displayed aggregate but never for an authoritative balance used to approve a debit. It also only warms keys that are actually read, which is correct because there is no point refreshing something nobody wants.
Remember one sentence: refresh-ahead keeps hot keys always fresh by reloading in the background before TTL expires, and probabilistic early recompute plus jitter prevent the synchronized stampede.
---