Definition
- Local state problem – a session in instance A's memory is invisible when the next request lands on instance B.
- Redis externalization – all instances read/write sessions from a shared Redis store.
- Stateless JWT – a signed token carries the identity in every request; no server-side session.
- Round robin – distributes equal request count but not equal work; a 5ms request and a 5-second report count the same.
- Least connections – routes to the instance with the fewest in-flight requests, accounting for actual load.
- Readiness health check – verifies the DB connection pool, dependencies, and the instance's real ability to serve.
National Bank of India has three identical counters and a floor manager sending customers to whichever is free. If teller A scribbles Rajesh's half-finished form on a sticky note on his own desk (local state), then Rajesh comes back and goes to teller B, teller B has never seen that note. The fix is a shared filing cabinet (Redis) or having Rajesh carry the form with him (JWT). The manager sending the same number of customers to each is unfair if some take 30 seconds and others take an hour. And checking that a teller is sitting there is not enough; you must check the computer and vault line work too.
Understanding the Concepts
Why does local state break under a load balancer?
With one instance, you could keep a session in memory. With multiple instances and a load balancer, consecutive requests from the same user may land on different instances. Instance A's session is invisible to instance B.
// WRONG: in-memory session map
private final Map<String, Session> sessions = new ConcurrentHashMap<>();
// Request 1 → instance A creates session
// Request 2 → instance B has no idea → user appears logged out
How do you fix it?
Option 1: Externalize to Redis
package com.bank.session;
import org.springframework.data.redis.core.StringRedisTemplate;
import java.time.Duration;
public class RedisSessionStore {
private final StringRedisTemplate redis;
public RedisSessionStore(StringRedisTemplate redis) {
this.redis = redis;
}
public void save(String sessionId, String json) {
redis.opsForValue().set("session:" + sessionId, json, Duration.ofMinutes(30));
}
public String find(String sessionId) {
return redis.opsForValue().get("session:" + sessionId);
}
}
Option 2: Stateless JWT
// JWT carries identity in every request; no server-side session
String token = Jwts.builder()
.setSubject("CUST-1001")
.claim("role", "customer")
.signWith(secretKey)
.compact();
// Any instance can validate and extract claims without shared state
Why is sticky sessions a weak fallback?
Sticky sessions pin a user to one instance, but if that instance dies the session is lost. It also undermines balancing because hot users pile onto one instance.
Why doesn't round robin guarantee equal work?
| Algorithm | Balances | Weakness |
|---|---|---|
| Round robin | Request count | Ignores request cost |
| Least connections | In-flight requests | Accounts for slow requests |
| Least response time | Actual speed | Routes to fastest instance |
// Round robin: instance A gets a 5-second report, instance B gets a 5ms lookup
// Both got 1 request, but A is overloaded and B is idle
// Least connections would route the NEXT request to B (fewer in-flight)
Why must health checks verify the instance can serve?
A naive liveness check confirms the process is running. But an instance can be up yet unable to serve: DB connection pool exhausted, downstream unreachable.
package com.bank.health;
import org.springframework.boot.actuate.health.Health;
import org.springframework.boot.actuate.health.HealthIndicator;
import org.springframework.stereotype.Component;
import javax.sql.DataSource;
@Component
public class ReadinessCheck implements HealthIndicator {
private final DataSource dataSource;
public ReadinessCheck(DataSource dataSource) {
this.dataSource = dataSource;
}
@Override
public Health health() {
try (var conn = dataSource.getConnection()) {
if (conn.isValid(1)) return Health.up().build();
} catch (Exception e) { /* pool exhausted */ }
return Health.down().build();
}
}
What are common mistakes?
// WRONG: health check only checks process is alive
@GetMapping("/health")
public String health() { return "OK"; } // says OK even with dead DB
// CORRECT: health check verifies DB and dependencies
@GetMapping("/health/ready")
public Health ready() {
return dbPool.isValid() ? Health.up().build() : Health.down().build();
}
What should a senior engineer know?
- Statelessness is the foundation of horizontal scaling; any local state is a scaling bug.
- Distinguish liveness (should orchestrator restart?) from readiness (should LB send traffic?).
- A readiness check that is too strict can cascade: if a shared dependency blips and all instances report unready, you take down the whole service.
- Match the balancing algorithm to request-cost variance.
Visual Overview
Local State Problem ┌──────┐ req 1 ┌──────────┐ session in memory │Rajesh│ ──────► │Instance A│ (only here!) └──────┘ └──────────┘ ┌──────┐ req 2 ┌──────────┐ │Rajesh│ ──────► │Instance B│ session NOT here → logged out └──────┘ └──────────┘ Fix: Externalize to Redis ┌──────┐ req 1 ┌──────────┐ save ┌───────┐ │Rajesh│ ──────► │Instance A│ ─────► │ Redis │ └──────┘ └──────────┘ └───────┘ ┌──────┐ req 2 ┌──────────┐ read ┌───────┐ │Rajesh│ ──────► │Instance B│ ─────► │ Redis │ ✓ session found └──────┘ └──────────┘ └───────┘ Health Check Levels Liveness: "Is the process alive?" → restart if no Readiness: "Can it actually serve?" → remove from LB if no
Key Takeaway
Putting a load balancer in front of multiple instances exposes local state as a bug. A session in one instance's memory is invisible when the next request lands on another instance. The fix is to externalize state to Redis so any instance can read it, or go fully stateless with a JWT that carries the identity in every request. Sticky sessions are a weak fallback that breaks when the pinned instance dies.
Round robin does not guarantee equal work because it balances request count, not cost. If one instance gets multi-second reports while another gets 5ms lookups, both got equal counts but unequal load. Least-connections routes to the instance with the fewest in-flight requests, naturally handling cost variance.
Health checks must verify that an instance can serve, not merely that it is running. A naive check that returns "OK" while the DB connection pool is exhausted is worse than no check, because the load balancer keeps sending traffic to an instance that fails every request. A readiness check that tests the connection pool and dependencies is what keeps the system honest.
Remember one sentence: externalize state to Redis or carry it in a JWT so requests scatter safely, use least connections because round robin balances count not work, and make health checks prove the instance can serve by testing its connection pool and dependencies.
---