Load Balancer

• Load balancer : a component that distributes incoming network traffic across a pool of backend servers to maximize throughput, minimize latency, and provide fault tolerance.
• Layer 4 (L4) : routes by transport-level information (IP address and port) without reading the request body; extremely fast and protocol-agnostic.
• Layer 7 (L7) : routes by application-level information (HTTP path, cookies, headers); enables content-based routing.
• Health check : a periodic probe that removes unhealthy backends from rotation until they recover.
• Distribution algorithm : the rule for choosing which backend gets the next request (round-robin, least-connections, IP hash).

What is a load balancer and why do we need it?

A load balancer distributes incoming network traffic across a pool of backend servers. Without one, you can only scale up a single server, which has a hard ceiling and is a single point of failure. With a load balancer you scale out across many servers and survive the loss of any one.

What is the difference between L4 and L7?

Layer 4 versus Layer 7 is about how much the balancer inspects. A Layer 4 balancer works at the transport layer and forwards packets based on IP address and port without reading the request body, which is extremely fast and protocol-agnostic. A Layer 7 balancer works at the application layer and can read the HTTP request, so it can route based on the URL path, cookies, or headers.

package com.bank.loadbalancer;

public class L7Router {
    // L7 routing: pick a backend pool based on the request URL
    public String route(String path) {
        if (path.startsWith("/api/transfers"))  return "transfer-pool";
        if (path.startsWith("/api/statements")) return "statement-pool";
        if (path.startsWith("/api/accounts"))   return "account-pool";
        return "default-pool";
    }
}

What are the common distribution algorithms?

Round-robin cycles through servers in order, which is simple and fair when servers are equal. Least-connections sends to the server with the fewest active connections, which is better when requests vary in duration. IP hash maps a client's IP to a consistent backend, giving session stickiness.

package com.bank.loadbalancer;

import java.util.List;
import java.util.Map;
import java.util.concurrent.atomic.AtomicInteger;

public class Algorithms {
    private final AtomicInteger counter = new AtomicInteger();

    public String roundRobin(List<String> servers) {
        return servers.get(counter.getAndIncrement() % servers.size());
    }

    public String leastConnections(Map<String, Integer> activeConns) {
        return activeConns.entrySet().stream()
            .min(Map.Entry.comparingByValue())
            .map(Map.Entry::getKey)
            .orElseThrow();
    }

    public String ipHash(String clientIp, List<String> servers) {
        int idx = Math.abs(clientIp.hashCode()) % servers.size();
        return servers.get(idx);   // same client always hits same server
    }
}

How do health checks work?

The balancer periodically probes each backend, often with an HTTP GET on a /health endpoint, and removes any server that fails from rotation until it recovers. This ensures clients never hit a dead server.

package com.bank.loadbalancer;

public class HealthChecker {
    public void check(List<Server> servers) {
        for (Server s : servers) {
            boolean healthy = httpClient.get(s.url() + "/health")
                .statusCode() == 200;
            s.setInRotation(healthy);  // remove if unhealthy
        }
    }
}

Should you use sticky sessions or stateless servers?

Sticky sessions via IP hash simplify stateful apps but hurt even distribution and complicate failover. The better pattern is to keep servers stateless and store session data in a shared cache like Redis, letting the balancer freely distribute requests to any healthy server.

package com.bank.loadbalancer;

public class SessionStrategy {
    // WRONG: sticky sessions (poor distribution, failover breaks)
    // If server-2 dies, all its sticky clients lose their sessions.

    // CORRECT: stateless servers + shared session store (Redis)
    // Any server can handle any request because session lives in Redis.
    public String getSession(String sessionId) {
        return redis.get("session:" + sessionId);
    }
}

How do you make the load balancer itself highly available?

The load balancer must not become a new single point of failure. It is deployed as a highly available pair (active-passive or active-active) using virtual IP failover, so if one balancer instance dies, the other takes over seamlessly.

What are common load balancer implementations?

Popular implementations include NGINX (L7, software), HAProxy (L4/L7, software), AWS ALB (L7, managed), AWS NLB (L4, managed), and cloud provider load balancers. A production stack often uses both: a fast L4 balancer in front for TLS and raw throughput, and an L7 balancer behind it for content-based routing.

What should a senior engineer know?

A senior engineer knows that a load balancer also terminates TLS, absorbs connection bursts, and provides a stable IP that DNS points to. They monitor backend response times at the balancer level to detect slow servers before they affect users, and they configure connection draining so in-flight requests finish before a server is taken out of rotation during deployments.