Design a Rate Limiter

• Rate limiter : controls how many requests a client may make in a time window, protecting systems from overload and abuse.
• Fixed window : counts requests per calendar interval; simple but has a boundary-doubling flaw.
• Sliding window : blends counts from current and previous windows for smooth rate enforcement.
• Token bucket : refills tokens at a steady rate up to a capacity; allows bursts while enforcing a long-term average.
• Distributed rate limiting : state in a shared store (Redis) with atomic operations so limits apply across all servers.

What is a rate limiter?

A rate limiter controls how many requests a client may make in a given time window, rejecting or delaying excess traffic. It protects systems from overload, abuse, and runaway cost.

How does fixed window work?

Fixed window divides time into intervals and counts requests per interval. It is simple but has a boundary flaw: a client can send the full limit at the end of one minute and the full limit at the start of the next.

package com.bank.ratelimit;

public class FixedWindowLimiter {
    public boolean allow(String client, int limit) {
        long minute = System.currentTimeMillis() / 60_000;
        String key = client + ":" + minute;
        int count = counters.merge(key, 1, Integer::sum);
        return count <= limit;
    }
}

How does sliding window fix the boundary?

Sliding window counter blends current and previous window counts weighted by position in the current window, smoothing the boundary.

How does token bucket work?

Token bucket models a bucket that refills at a steady rate up to a capacity. Each request removes one token; empty means rejected. This allows short bursts up to capacity while enforcing a long-term average.

package com.bank.ratelimit;

public class TokenBucketLimiter {
    public synchronized boolean allow(String client, double rate, double capacity) {
        Bucket b = buckets.computeIfAbsent(client,
            k -> new Bucket(capacity, System.nanoTime()));
        long now = System.nanoTime();
        double elapsed = (now - b.lastRefill) / 1e9;
        b.tokens = Math.min(capacity, b.tokens + elapsed * rate);
        b.lastRefill = now;
        if (b.tokens >= 1) { b.tokens -= 1; return true; }
        return false;
    }
}

How do you distribute rate limiting?

In a multi-server setup, an attacker can spread requests across servers. State must live in Redis with atomic Lua scripts so the read-compute-write is one operation.

package com.bank.ratelimit;

public class DistributedLimiter {
    // Lua script runs atomically in Redis
    // KEYS[1] = client key, ARGV[1] = limit, ARGV[2] = window seconds
    // Returns 1 if allowed, 0 if throttled
    private static final String LUA_SCRIPT =
        "local count = redis.call('INCR', KEYS[1]) " +
        "if count == 1 then redis.call('EXPIRE', KEYS[1], ARGV[2]) end " +
        "return count <= tonumber(ARGV[1]) and 1 or 0";
}

Where should the limiter be placed?

At the API gateway, so bad traffic is rejected at the front door before consuming downstream resources. Return HTTP 429 with a Retry-After header.

How do you configure per-endpoint limits?

Different endpoints have different costs. A cheap balance read can tolerate a higher rate than an expensive transfer.

package com.bank.ratelimit;

public class EndpointLimits {
    public int limitFor(String endpoint) {
        return switch (endpoint) {
            case "/api/balance" -> 100;   // cheap, high limit
            case "/api/transfer" -> 10;   // expensive, low limit
            case "/api/login" -> 5;       // security-sensitive
            default -> 50;
        };
    }
}

What should a senior engineer know?

A senior engineer considers rate limiting by multiple dimensions (IP, API key, user), implements graceful degradation rather than hard rejection for internal services, and uses rate limiting for cost control in addition to security.