Definition
- Problem – a per-instance local counter breaks with multiple instances because each instance allows the full limit independently.
- Solution – key the limit by user ID or API key and store the counter in a shared, atomic store like Redis.
- Atomicity – use a Lua script or Bucket4j with JCache so read-check-consume is one indivisible operation.
- Algorithm – token bucket (permits bursts) or leaky bucket (smooth constant outflow) stored in Redis.
- Enforcement point – the API gateway rejects excess with HTTP 429 before backends consume capacity.
- Production concerns – Redis must be highly available, layer per-user with per-endpoint limits, and fail open if Redis is briefly unavailable.
Imagine the National Bank of India lets each customer submit at most ten forms per minute, but the bank has five branches and a customer might walk into any of them. If each branch kept its own private tally, the customer could submit ten forms at every branch and sneak through fifty. So instead the bank keeps one central ledger, shared by all branches, that records how many forms each customer has submitted this minute. Every branch checks and updates that one central ledger before accepting a form. That central shared ledger is Redis.
Understanding the Concepts
What is the distributed rate limiting problem?
A naive rate limiter stores counters in the memory of a single instance. This breaks the moment you run more than one instance.
package com.bank.ratelimit;
import java.util.concurrent.atomic.AtomicInteger;
// WRONG: Per-instance counter. 5 instances = 5x the intended limit.
public class LocalRateLimiter {
private final AtomicInteger count = new AtomicInteger(0);
private final int limit = 100;
public boolean isAllowed(String userId) {
return count.incrementAndGet() <= limit;
}
}
Problem:
Limit: 100 requests/minute per user
5 gateway instances, each with local counter
Effective limit: 5 x 100 = 500 requests/minute per user
A scripted client easily bypasses the intended limit.
How does Redis solve the problem?
The counter is stored in Redis, keyed by user ID. Every instance reads and updates the same shared counter.
package com.bank.ratelimit;
import redis.clients.jedis.Jedis;
import java.util.logging.Logger;
public class RedisRateLimiter {
private static final Logger LOG = Logger.getLogger(RedisRateLimiter.class.getName());
private static final int LIMIT = 100;
private static final int WINDOW_SECONDS = 60;
private final Jedis jedis;
public RedisRateLimiter(Jedis jedis) {
this.jedis = jedis;
}
public boolean isAllowed(String userId) {
String key = "rate:user:" + userId + ":transfers";
Long current = jedis.incr(key);
if (current == 1) {
jedis.expire(key, WINDOW_SECONDS);
}
boolean allowed = current <= LIMIT;
if (!allowed) {
LOG.warning("Rate limit exceeded for user: " + userId);
}
return allowed;
}
}
Expected behavior:
All 5 gateway instances increment the SAME Redis key.
User SAV2001 sends request 100 -> allowed (count=100)
User SAV2001 sends request 101 -> rejected (count=101, returns 429)
Why must the update be atomic?
A naive read-then-write has a race condition. Two instances can both read the same count and both allow a request.
-- Atomic Lua script for token bucket in Redis
-- Runs as one indivisible server-side operation
local key = KEYS[1]
local capacity = tonumber(ARGV[1])
local refillRate = tonumber(ARGV[2])
local now = tonumber(ARGV[3])
local data = redis.call('HMGET', key, 'tokens', 'lastRefill')
local tokens = tonumber(data[1]) or capacity
local lastRefill = tonumber(data[2]) or now
-- Refill tokens based on elapsed time
local elapsed = now - lastRefill
local newTokens = math.min(capacity, tokens + elapsed * refillRate)
if newTokens >= 1 then
redis.call('HMSET', key, 'tokens', newTokens - 1, 'lastRefill', now)
redis.call('EXPIRE', key, 120)
return 1 -- allowed
else
return 0 -- rejected
end
WRONG: Two-step read-then-write
Instance A reads count=99 -> allows
Instance B reads count=99 -> allows
Both write count=100 -> user got 101 requests through
CORRECT: Lua script (atomic)
Read + check + consume in one Redis operation
No race condition possible
How do you implement with Bucket4j?
Bucket4j is the standard Java library for token-bucket rate limiting. It integrates with Redis via Redisson for distributed buckets.
package com.bank.ratelimit;
import io.github.bucket4j.Bandwidth;
import io.github.bucket4j.Bucket;
import io.github.bucket4j.BucketConfiguration;
import io.github.bucket4j.distributed.proxy.ProxyManager;
import java.time.Duration;
import java.util.function.Supplier;
import java.util.logging.Logger;
public class Bucket4jRateLimiter {
private static final Logger LOG = Logger.getLogger(Bucket4jRateLimiter.class.getName());
private final ProxyManager<String> proxyManager; // Redis-backed via Redisson
public Bucket4jRateLimiter(ProxyManager<String> proxyManager) {
this.proxyManager = proxyManager;
}
public boolean tryConsume(String userId) {
Supplier<BucketConfiguration> config = () -> BucketConfiguration.builder()
.addLimit(Bandwidth.simple(100, Duration.ofMinutes(1))) // 100 req/min
.build();
Bucket bucket = proxyManager.builder()
.build("rate:user:" + userId, config);
boolean allowed = bucket.tryConsume(1);
if (!allowed) {
LOG.warning("Bucket4j: rate limit hit for user " + userId);
}
return allowed;
}
}
Expected behavior:
User SAV2001 calls tryConsume -> true (tokens: 99 remaining)
... 99 more calls -> true (tokens: 0)
101st call -> false (bucket empty, return 429)
After 1 minute: bucket refills to 100 tokens.
How do you enforce at the gateway?
Spring Cloud Gateway has a built-in Redis-backed RequestRateLimiter with a key resolver that extracts the user ID.
package com.bank.gateway;
import org.springframework.cloud.gateway.filter.ratelimit.KeyResolver;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import reactor.core.publisher.Mono;
@Configuration
public class RateLimiterConfig {
@Bean
public KeyResolver userKeyResolver() {
// Extract user ID from JWT claims or API key header
return exchange -> Mono.just(
exchange.getRequest().getHeaders()
.getFirst("X-User-Id"));
}
}
# Gateway route with Redis rate limiter
spring:
cloud:
gateway:
routes:
- id: transfer-route
uri: lb://payment-service
predicates:
- Path=/api/v1/transfers/**
filters:
- name: RequestRateLimiter
args:
redis-rate-limiter.replenishRate: 10
redis-rate-limiter.burstCapacity: 20
key-resolver: "#{@userKeyResolver}"
Expected behavior:
Replenish rate: 10 tokens/second per user
Burst capacity: 20 (allows burst then settles to 10/sec)
Excess: HTTP 429 with Retry-After header
All gateway instances share the same Redis bucket per user.
How do token bucket and leaky bucket compare?
| Aspect | Token Bucket | Leaky Bucket |
|---|---|---|
| Burst behavior | Permits bursts up to capacity | Smooths to constant outflow |
| Implementation | Refill tokens at steady rate | Process at constant rate |
| Use case | Allow bursts (e.g., page load) | Strict smoothing |
| Redis storage | Tokens + last refill timestamp | Queue position |
What are common mistakes?
The biggest mistake is per-instance counters in a distributed environment.
package com.bank.antipattern;
// WRONG: Per-instance counter. Useless with multiple gateway instances.
public class WrongLimiter {
private int count = 0;
public boolean allow() { return ++count <= 100; }
}
// CORRECT: Shared Redis counter with atomic Lua script.
// All instances share one bucket per user in Redis.
// Bucket4j + Redisson or Spring Cloud Gateway RedisRateLimiter.
What should a senior engineer know?
Return 429 with Retry-After and rate-limit headers so clients back off gracefully. Redis is on the critical path so make it highly available. Layer a per-user limit with a stricter per-endpoint limit for expensive operations. Design a fallback: fail open or drop to a conservative local limit if Redis is briefly unavailable.
Visual Overview
Per-User Distributed Rate Limiting:
┌──────────────────────────────────────────┐
│ Gateway Instance 1 │
│ User SAV2001 -> check Redis bucket │
└────────────────────┬─────────────────────┘
│
┌────────────────────┴─────────────────────┐
│ REDIS │
│ rate:user:SAV2001 -> {tokens:47, ts:...}│
│ Atomic Lua: refill + check + consume │
│ Single source of truth │
└────────────────────┬─────────────────────┘
│
┌────────────────────┴─────────────────────┐
│ Gateway Instance 2 │
│ User SAV2001 -> check SAME Redis bucket │
└──────────────────────────────────────────┘
Both instances consume from the SAME bucket.
Limit holds across the entire fleet.
Key Takeaway
Per-user rate limiting in a distributed environment comes down to one central idea: the counter must be shared and updated atomically, because a limiter that keeps state in a single instance's memory breaks the moment you run more than one instance. If the limit is a hundred requests per minute per user and you run five instances, each allows a hundred and the user effectively gets five hundred.
The solution is to key the limit by user ID, store the counter in Redis, and update it atomically with a Lua script or Bucket4j backed by Redisson. The token bucket algorithm permits controlled bursts up to the bucket capacity while enforcing a steady long-run average rate. At National Bank of India, Pankaj enforces the limit at the API gateway using Spring Cloud Gateway's built-in Redis-backed RequestRateLimiter, so excess is rejected with HTTP 429 before any backend capacity is consumed.
The senior-level nuances are that Redis is on the critical path so it must be highly available, you should layer a per-user limit with a per-endpoint limit for expensive operations, and you should design a fallback that fails open or drops to a conservative local limit if Redis is briefly unavailable.
Remember one sentence: key by user, store the bucket in Redis, consume atomically with a Lua script, enforce at the gateway, and the limit holds globally across every instance.
---