Definition
- Round Robin – cycles through instances in order, ideal for uniform requests and equal instances, but ignores actual load.
- Least Connections – sends each request to the instance with the fewest active connections, adapting when request durations vary.
- Least Response Time – picks the instance with the lowest observed latency, best when instances differ in speed.
- IP Hash / Sticky Sessions – pins a client to a consistent instance, needed when the instance holds client-specific local state.
- Weighted – assigns more traffic to more powerful instances, used for heterogeneous fleets and canary rollouts.
- Consistent Hashing – maps instances and keys onto a hash ring so adding or removing an instance remaps only a small fraction of keys, essential for distributed caches.
Imagine the floor manager at National Bank of India assigning arriving customers to teller windows. Round robin is "next customer to the next window in order." Least connections is "send the next customer to the window with the shortest queue right now." IP hash is "Mr. Rajesh always goes to window three because that teller holds his half-finished paperwork." Consistent hashing is a smart seating chart where opening or closing one window only reshuffles a few customers instead of the whole hall.
Understanding the Concepts
What is round robin load balancing?
Round robin distributes requests by cycling through the instance list in order: first request to instance A, second to B, third to C, then back to A. It is simple, stateless, and fair when every request costs about the same and every instance is equally powerful.
package com.bank.lb;
import java.util.List;
import java.util.concurrent.atomic.AtomicInteger;
public class RoundRobinBalancer {
private final List<String> instances;
private final AtomicInteger index = new AtomicInteger(0);
public RoundRobinBalancer(List<String> instances) {
this.instances = instances;
}
public String next() {
int i = index.getAndIncrement();
return instances.get(i % instances.size());
}
public static void main(String[] args) {
var lb = new RoundRobinBalancer(List.of("account-svc-1", "account-svc-2", "account-svc-3"));
for (int r = 0; r < 6; r++) {
System.out.println("Request " + r + " → " + lb.next());
}
}
}
Request 0 → account-svc-1 Request 1 → account-svc-2 Request 2 → account-svc-3 Request 3 → account-svc-1 Request 4 → account-svc-2 Request 5 → account-svc-3
What is least connections load balancing?
Least connections sends each new request to the instance currently handling the fewest active connections. This adapts to reality: if one instance is bogged down by long-running requests, its connection count stays high and it receives fewer new requests.
package com.bank.lb;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.atomic.AtomicInteger;
public class LeastConnectionsBalancer {
private final Map<String, AtomicInteger> connections = new ConcurrentHashMap<>();
public LeastConnectionsBalancer(java.util.List<String> instances) {
instances.forEach(i -> connections.put(i, new AtomicInteger(0)));
}
public String acquire() {
return connections.entrySet().stream()
.min(Map.Entry.comparingByValue(
java.util.Comparator.comparingInt(AtomicInteger::get)))
.map(e -> { e.getValue().incrementAndGet(); return e.getKey(); })
.orElseThrow();
}
public void release(String instance) {
connections.get(instance).decrementAndGet();
}
public static void main(String[] args) {
var lb = new LeastConnectionsBalancer(java.util.List.of("svc-1", "svc-2", "svc-3"));
String first = lb.acquire();
String second = lb.acquire();
System.out.println("First request → " + first);
System.out.println("Second request → " + second + " (different, both had 0 before)");
// svc-1 now has 1 connection, svc-2 has 1, svc-3 has 0
String third = lb.acquire();
System.out.println("Third request → " + third + " (least connections: 0)");
}
}
First request → svc-1 Second request → svc-2 (different, both had 0 before) Third request → svc-3 (least connections: 0)
What is IP hash and sticky sessions?
IP hash computes a hash of the client's IP and maps it to an instance, so the same client consistently lands on the same one. Sticky sessions achieve the same using a cookie. You use these only when the instance holds client-specific local state such as an in-memory session.
package com.bank.lb;
import java.util.List;
public class IpHashBalancer {
private final List<String> instances;
public IpHashBalancer(List<String> instances) {
this.instances = instances;
}
public String route(String clientIp) {
int hash = Math.abs(clientIp.hashCode());
return instances.get(hash % instances.size());
}
public static void main(String[] args) {
var lb = new IpHashBalancer(List.of("svc-1", "svc-2", "svc-3"));
// Same IP always routes to the same instance
System.out.println("Rajesh (10.0.0.1) → " + lb.route("10.0.0.1"));
System.out.println("Rajesh (10.0.0.1) → " + lb.route("10.0.0.1"));
System.out.println("Priya (10.0.0.2) → " + lb.route("10.0.0.2"));
}
}
Rajesh (10.0.0.1) → svc-2 Rajesh (10.0.0.1) → svc-2 Priya (10.0.0.2) → svc-3
What is consistent hashing?
Consistent hashing maps both instances and request keys onto a hash ring. Each key is served by the next instance clockwise on the ring. When you add or remove an instance, only the keys in that arc are remapped, not all keys.
package com.bank.lb;
import java.util.SortedMap;
import java.util.TreeMap;
public class ConsistentHashRing {
private final SortedMap<Integer, String> ring = new TreeMap<>();
private static final int VIRTUAL_NODES = 100;
public void addInstance(String instance) {
for (int i = 0; i < VIRTUAL_NODES; i++) {
ring.put((instance + "-vn" + i).hashCode(), instance);
}
}
public void removeInstance(String instance) {
for (int i = 0; i < VIRTUAL_NODES; i++) {
ring.remove((instance + "-vn" + i).hashCode());
}
}
public String route(String key) {
int hash = key.hashCode();
SortedMap<Integer, String> tail = ring.tailMap(hash);
int target = tail.isEmpty() ? ring.firstKey() : tail.firstKey();
return ring.get(target);
}
public static void main(String[] args) {
var ring = new ConsistentHashRing();
ring.addInstance("cache-1");
ring.addInstance("cache-2");
ring.addInstance("cache-3");
System.out.println("SAV2001 → " + ring.route("SAV2001"));
System.out.println("CUR3005 → " + ring.route("CUR3005"));
// Adding cache-4 remaps only ~25% of keys, not all
ring.addInstance("cache-4");
System.out.println("After adding cache-4:");
System.out.println("SAV2001 → " + ring.route("SAV2001"));
}
}
SAV2001 → cache-2 CUR3005 → cache-1 After adding cache-4: SAV2001 → cache-2
How do you compare all the algorithms?
| Algorithm | Best For | State Needed | Weakness |
|---|---|---|---|
| Round Robin | Uniform requests, equal instances | None (index only) | Load-blind |
| Least Connections | Variable request durations | Live connection counts | Slightly more complex |
| Least Response Time | Heterogeneous instances | Per-instance latency | Latency measurement overhead |
| IP Hash / Sticky | Client-specific local state | None (hash function) | Uneven distribution, breaks on death |
| Weighted | Unequal capacity, canary | Weight config | Manual weight tuning |
| Consistent Hashing | Caches, shards, dynamic fleet | Hash ring | Complexity, needs virtual nodes |
What are common mistakes?
The most common mistake is using sticky sessions when the real fix is to externalize state. Stickiness undermines even load distribution and breaks if that instance dies.
package com.bank.lb;
// WRONG: storing session in-memory and relying on sticky sessions
public class WrongStickyApproach {
// Each instance holds user sessions in a local HashMap
// If the instance dies, the session is lost
// Sticky sessions mask the real problem: state should be external
}
// CORRECT: externalize state to Redis, no stickiness needed
public class CorrectStatelessApproach {
// Sessions stored in shared Redis
// Any instance can serve any request
// Load balancer uses round robin or least connections freely
}
Why must every algorithm include health checks?
Distributing traffic evenly to a dead instance is worse than useless. Every algorithm must be combined with health checks so dead instances are removed from rotation.
package com.bank.lb;
import java.util.List;
import java.util.ArrayList;
public class HealthAwareBalancer {
private final List<String> allInstances;
public HealthAwareBalancer(List<String> instances) {
this.allInstances = new ArrayList<>(instances);
}
public List<String> getHealthy() {
// In production: periodic HTTP health check to each instance
return allInstances.stream()
.filter(i -> !i.equals("svc-2")) // svc-2 failed health check
.toList();
}
public static void main(String[] args) {
var lb = new HealthAwareBalancer(List.of("svc-1", "svc-2", "svc-3"));
System.out.println("Healthy instances: " + lb.getHealthy());
System.out.println("svc-2 removed from rotation after failed health check");
}
}
Healthy instances: [svc-1, svc-3]
svc-2 removed from rotation after failed health check
What are common interview follow-up questions?
- Why do virtual nodes improve consistent hashing distribution?
- How does weighted round robin work during a canary deployment?
- What is the difference between L4 and L7 load balancing?
- When does least response time outperform least connections?
- How do you handle session affinity in a Kubernetes Ingress?
Visual Overview
Consistent Hashing Ring
────────────────────────
0
┌────┴────┐
/ \
cache-3 cache-1
│ │
│ hash ring │
│ │
cache-2 cache-4
\ /
└────┬────┘
512
Adding cache-4 remaps only keys in its arc (~25%).
Removing cache-2 remaps only its keys to cache-3.
Virtual nodes (100 per instance) smooth the distribution.
Algorithm Decision Tree
───────────────────────
Is it a cache or sharded store? ──Yes──► Consistent Hashing
│
No
│
Does the instance hold client state? ──Yes──► IP Hash / Sticky
│ (but externalize state!)
No
│
Are instances unequal in capacity? ──Yes──► Weighted Round Robin
│
No
│
Do request durations vary widely? ──Yes──► Least Connections
│
No
│
► Round Robin (simplest default)
Key Takeaway
Load-balancing algorithms decide which backend instance serves each request, and the right choice follows the workload. Round robin simply cycles through instances in order and is fair when requests are uniform and instances are equal, but it is load-blind. Least connections sends each request to the instance with the fewest active connections, which adapts naturally when request durations vary. Least response time routes toward the fastest instance, best for heterogeneous fleets.
IP hash and sticky sessions pin a client to a consistent instance, but stickiness is a smell: it undermines even distribution and breaks on instance death, so the better fix is to externalize state. Weighted variants assign more traffic to more powerful instances, useful for canary deployments. Consistent hashing maps instances and keys onto a ring so adding or removing a node remaps only a small fraction of keys, which is essential for distributed caches and sharded stores.
Every algorithm must be paired with health checks so a dead instance leaves rotation immediately. The senior judgment is to match the algorithm to the workload: round robin for uniform requests, least connections as a sensible default, consistent hashing for caches and shards, and stickiness only as a temporary measure until state is externalized.
Remember one sentence: round robin for uniform equal instances, least connections for variable durations, consistent hashing for caches, and always pair with health checks.
---