Definition
- Sticky routing – a Layer-4 TCP load balancer or Service with
sessionAffinity: ClientIPpins a client's persistent connection to one pod for its lifetime. - Consistent-hash routing – the same client key prefers the same pod across reconnects, reducing state disruption.
- Client-side LB –
LoadbalanceRSocketClienton the client discovers instances and balances connections across them without a proxy. - HPA on connections – Horizontal Pod Autoscaler driven by custom metrics like active connection count per pod, exported via Micrometer to Prometheus.
- Resumable sessions – resume tokens let a dropped connection reconnect and replay unacknowledged frames so streams survive pod churn.
- Keep-alive tuning – balance fast dead-peer detection against tolerance for brief rolling-update pauses.
Imagine National Bank of India opening more trading-desk counters as traffic grows. Because each customer like Rajesh is on a long live phone call, you cannot randomly bounce each sentence to a different clerk; you must keep his whole call pinned to one clerk (sticky routing). When you open new counters, you decide based on how many live calls are active (connections), not how many one-off questions arrive. If a clerk's counter closes for maintenance, Rajesh's call automatically redials and resumes where it left off (resumable session), and the heartbeat check on the line must be patient enough not to hang up during the brief handover (keep-alive tuning).
Understanding the Concepts
Why is scaling RSocket different from scaling HTTP?
The core challenge is that RSocket connections are long-lived and stateful. All frames of one connection and its streams must reach the same pod. A standard HTTP Layer-7 load balancer that re-routes each request independently does not work for RSocket.
HTTP (stateless): Request 1 ──▶ Pod A Request 2 ──▶ Pod B ← different pod, no problem Request 3 ──▶ Pod C RSocket (stateful connection): Connection ──▶ Pod A Frame 1 ──▶ Pod A ← same pod, always Frame 2 ──▶ Pod A Stream ──▶ Pod A
How does sticky load balancing work?
Use a Layer-4 TCP load balancer or a Kubernetes Service with session affinity to pin a client's connection to one backend pod.
# Kubernetes Service with session affinity
apiVersion: v1
kind: Service
metadata:
name: rsocket-price-service
namespace: bank-trading
spec:
type: LoadBalancer
sessionAffinity: ClientIP
sessionAffinityConfig:
clientIP:
timeoutSeconds: 3600 # 1 hour sticky session
selector:
app: rsocket-price-server
ports:
- port: 7000
targetPort: 7000
protocol: TCP
package com.bank.rsocket;
import io.rsocket.core.RSocketConnector;
import io.rsocket.transport.netty.client.TcpClientTransport;
public class StickyClient {
public static void main(String[] args) {
// Client connects to the Kubernetes Service VIP
// Session affinity pins this connection to one pod
RSocketConnector.create()
.connect(TcpClientTransport.create(
"rsocket-price-service.bank-trading", 7000))
.subscribe(rSocket ->
System.out.println("Connected to pod (sticky)"));
}
}
How does client-side load balancing work?
The RSocket client itself discovers server instances and balances connections across them, removing the need for a smart proxy.
package com.bank.rsocket;
import io.rsocket.core.RSocketConnector;
import io.rsocket.loadbalance.LoadbalanceRSocketClient;
import io.rsocket.loadbalance.LoadbalanceTarget;
import io.rsocket.transport.netty.client.TcpClientTransport;
import reactor.core.publisher.Flux;
import java.util.List;
public class ClientSideLB {
public static void main(String[] args) {
// Discover server instances (from Kubernetes DNS or registry)
Flux<List<LoadbalanceTarget>> servers = Flux.just(List.of(
LoadbalanceTarget.from("pod-1",
TcpClientTransport.create("10.0.0.1", 7000)),
LoadbalanceTarget.from("pod-2",
TcpClientTransport.create("10.0.0.2", 7000)),
LoadbalanceTarget.from("pod-3",
TcpClientTransport.create("10.0.0.3", 7000))
));
LoadbalanceRSocketClient client =
LoadbalanceRSocketClient.builder(servers).build();
// Client balances across pods
client.requestResponse(
io.rsocket.util.DefaultPayload.create("SAV2001"))
.subscribe(payload -> System.out.println(
"Balance: " + payload.getDataUtf8()));
}
}
How do you autoscale on connection metrics?
Request-per-second is the wrong signal because one connection can carry a long stream with little request churn. Use active connection count or event-loop utilization.
package com.bank.rsocket;
import io.micrometer.core.instrument.Gauge;
import io.micrometer.core.instrument.MeterRegistry;
import java.util.concurrent.atomic.AtomicInteger;
public class ConnectionMetrics {
private final AtomicInteger activeConnections = new AtomicInteger(0);
public void registerMetrics(MeterRegistry registry) {
// Export to Prometheus for HPA
Gauge.builder("rsocket.connections.active",
activeConnections, AtomicInteger::get)
.description("Active RSocket connections")
.tag("service", "price-service")
.register(registry);
}
public void onConnect() {
int count = activeConnections.incrementAndGet();
System.out.println("Connection opened, active: " + count);
}
public void onDisconnect() {
int count = activeConnections.decrementAndGet();
System.out.println("Connection closed, active: " + count);
}
}
# HPA based on active connections
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: rsocket-price-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: rsocket-price-server
minReplicas: 2
maxReplicas: 10
metrics:
- type: Pods
pods:
metric:
name: rsocket_connections_active
target:
type: AverageValue
averageValue: "500" # Scale when avg connections > 500
behavior:
scaleDown:
stabilizationWindowSeconds: 300 # generous cooldown for sticky connections
How do resumable sessions handle pod churn?
Pods are routinely rescheduled during deploys, node drains, and autoscaling. With resume enabled, a client whose pod disappears reconnects and continues its streams.
package com.bank.rsocket;
import io.rsocket.core.RSocketConnector;
import io.rsocket.core.Resume;
import io.rsocket.transport.netty.client.TcpClientTransport;
import reactor.util.retry.Retry;
import java.time.Duration;
public class ResumeForPodChurn {
public static void main(String[] args) {
RSocketConnector.create()
.resume(new Resume()
.sessionDuration(Duration.ofMinutes(5))
.retry(Retry.backoff(10, Duration.ofSeconds(1))))
.connect(TcpClientTransport.create(
"rsocket-price-service.bank-trading", 7000))
.flatMapMany(rSocket ->
rSocket.requestStream(
io.rsocket.util.DefaultPayload.create("RELIANCE")))
.doOnNext(p -> System.out.println(
"Price: " + p.getDataUtf8()))
.subscribe();
// If the pod is rescheduled during a rolling update,
// the client reconnects, presents the resume token,
// and the stream continues from where it left off.
}
}
How do you tune keep-alive for rolling updates?
Set keep-alive intervals to be patient enough to survive a brief rolling-update pause while still detecting genuinely dead pods.
// Keep-alive tuning for Kubernetes rolling updates
RSocketConnector.create()
.keepAlive(
Duration.ofSeconds(30), // ping every 30s
Duration.ofSeconds(120)) // declare dead after 120s
// A rolling update's preStop + terminationGracePeriod
// should complete within 120 seconds
# Pod spec with graceful drain
spec:
terminationGracePeriodSeconds: 90
containers:
- name: rsocket-server
lifecycle:
preStop:
exec:
command: ["/bin/sh", "-c", "sleep 10"] # drain connections
What are common mistakes when scaling RSocket?
The most common mistake is using a standard HTTP load balancer that distributes frames from one connection to different pods.
// WRONG: Layer-7 HTTP LB re-routes individual frames Frame 1 ──▶ Pod A Frame 2 ──▶ Pod B ← connection state is on Pod A! Frame 3 ──▶ Pod C ← stream broken // CORRECT: Layer-4 TCP LB or client-side LB All frames ──▶ Pod A (for the lifetime of the connection)
What should a senior engineer know?
- Set
terminationGracePeriodSecondsand a preStop hook long enough to let active streams drain. - Use readiness gates so new connections only go to ready pods.
- Scale-in can strand connections on a terminating pod, so drain and let clients reconnect to remaining pods.
- Prefer graceful connection draining on shutdown over abrupt termination.
package com.bank.rsocket;
import io.rsocket.SocketAcceptor;
import io.rsocket.core.RSocketServer;
import io.rsocket.transport.netty.server.TcpServerTransport;
import reactor.core.Disposable;
public class GracefulShutdown {
private Disposable server;
public void start() {
server = RSocketServer.create(
SocketAcceptor.forRequestResponse(payload ->
reactor.core.publisher.Mono.just(
io.rsocket.util.DefaultPayload.create("OK"))))
.bind(TcpServerTransport.create(7000))
.subscribe();
}
public void shutdown() {
// Graceful: stop accepting new connections,
// drain existing streams, then dispose
System.out.println("Draining connections...");
server.dispose();
System.out.println("Server shut down gracefully");
}
}
Visual Overview
Scaling RSocket in Kubernetes:
Option 1: Sticky LB
┌────────┐ ┌───────────────┐ ┌───────┐
│ Client │─────▶│ L4 TCP LB │─────▶│ Pod A │
│(Rajesh)│ │ (sticky) │ │(pinned│
└────────┘ └───────────────┘ │ here) │
└───────┘
Option 2: Client-Side LB
┌────────┐ ┌───────┐
│ Client │─────▶│ Pod A │
│ │─────▶│ Pod B │ (client balances)
│ │─────▶│ Pod C │
└────────┘ └───────┘
Autoscaling:
┌──────────────────────────────────────┐
│ Prometheus │
│ rsocket_connections_active = 600 │
│ │ │
│ ▼ │
│ HPA: avg > 500 → scale to 3 replicas │
└──────────────────────────────────────┘
Rolling Update with Resume:
Pod A (old) ──▶ draining ──▶ terminated
Client ──resume token──▶ Pod B (new)
Stream continues from where it left off
Key Takeaway
Scaling RSocket in Kubernetes starts with accepting that RSocket connections are long-lived and stateful, so you cannot treat it like stateless HTTP where each request is independently routed. All frames of one connection and its streams must reach the same pod. Achieve that either with a Layer-4 TCP load balancer or Service with client-IP session affinity, or with client-side load balancing where the RSocket client itself discovers and balances connections across instances.
For autoscaling, drive the Horizontal Pod Autoscaler on connection-oriented metrics, not request rate, because one connection can carry a long stream with very little request churn. Export an active-connections gauge through Micrometer to Prometheus and scale out when average connections per pod cross a threshold. Keep the scale-down stabilization window generous so sticky connections are not torn down abruptly.
Resumable sessions are essential because pods get rescheduled constantly during deploys, node drains, and autoscaling. With resume enabled, a client whose pod disappears reconnects and continues its streams instead of erroring. Tune keep-alive to be patient enough to survive a rolling-update handover while still detecting genuinely dead pods, and set a long enough termination grace period with a preStop drain hook so active streams can finish.
Remember one sentence: pin connections with sticky or client-side load balancing, autoscale on connection metrics, enable resumption for pod churn, and tune keep-alive so rolling updates do not drop healthy traders.
---