How an API Gateway handles caching and what problems arise | Microservices

Q: What are common mistakes? (wrong vs correct)

// WRONG: caching POST requests // POST /api/v1/payments/transfer → cached response! // Next POST returns the cached transfer result // Customer charged once but sees two transfer confirmations // CORRECT: only cache GET (safe, idempotent methods) // POST, PUT, DELETE are never cached at the gateway

Definition

Cache safe, shared, slowly changing reads at the gateway with a sensible TTL, and never cache per-user or sensitive data without strict per-identity keys and short expiry.

Gateway caching – stores responses of safe, repeatable GET requests at the edge so subsequent identical requests skip the backend.
Cache key – must include method, path, query, and identity for personalized data; too coarse leaks data, too fine never hits.
TTL – time-to-live controls how long a cached response is served; anything cached can be stale for up to this duration.
Stale data – the fundamental trade-off; after Rajesh transfers money, a cached balance shows the wrong number.
Active invalidation – the service tells the gateway to evict a key when the underlying data changes.
Sensitive data risk – caching user-specific responses without identity in the key leaks one user's data to another.

Imagine the National Bank of India front desk keeps a printed sheet of today's fixed-deposit interest rates. When any customer asks, the clerk hands over the sheet instantly instead of phoning the treasury department. That is caching. If the rates changed this morning and the sheet is from yesterday, the clerk gives wrong numbers, that is stale data. If the clerk accidentally keeps a copy of Rajesh's private balance on the shared desk and hands it to the next person, that is the disaster of caching sensitive per-user data.

Understanding the Concepts

How does gateway caching work?

The gateway caches at the HTTP layer. It only caches idempotent, safe methods, almost always GET. Spring Cloud Gateway provides a LocalResponseCache filter that stores the response body and headers keyed by the request.

YAML

# Spring Cloud Gateway local response cache
spring:
  cloud:
    gateway:
      routes:
        - id: interest-rates
          uri: lb://rates-service
          predicates:
            - Path=/api/v1/rates/**
          filters:
            - name: LocalResponseCache
              args:
                timeToLive: 5m
                size: 100MB

DIAGRAM

// First request: GET /api/v1/rates/fd → calls rates-service → caches response
// Next 1000 requests within 5 minutes → served from cache → no backend call
// After 5 minutes → cache expires → next request fetches fresh data

Why is the cache key critical for correctness?

A key that is too coarse serves one user's data to another. A key that is too fine never gets a hit. The right key includes method, path, query, and for personalized responses, the user identity.

JAVA

package com.bank.gateway;

public class CacheKeyExamples {

    // WRONG: key is only the path
    // GET /api/v1/accounts/balance → key: "/api/v1/accounts/balance"
    // Rajesh's balance cached → Priya gets Rajesh's balance!

    // CORRECT: key includes user identity
    // GET /api/v1/accounts/balance (User: Rajesh)
    //   → key: "GET:/api/v1/accounts/balance:user=Rajesh"
    // GET /api/v1/accounts/balance (User: Priya)
    //   → key: "GET:/api/v1/accounts/balance:user=Priya"
    // Different keys → different cached responses → no cross-user leak

    // WRONG: key includes timestamp or request ID
    // key: "GET:/api/v1/rates/fd:ts=1705312800123"
    // Every request has a unique timestamp → cache never hits → useless
}

What is the stale data problem?

Anything cached can be out of date for up to its TTL. Right after Rajesh transfers money, a cached balance shows the old number.

JAVA

// Timeline:
// T=0:  Rajesh's balance = 75000 → cached with TTL 30s
// T=5:  Rajesh transfers 50000 → actual balance = 25000
// T=10: Rajesh checks balance → cache returns 75000 (stale!)
// T=30: Cache expires → next check returns 25000 (fresh)

// For 25 seconds, Rajesh sees the wrong balance
// In banking, this confuses customers and erodes trust

JAVA

// Fix 1: Very short TTL for volatile data
// Balance: TTL = 5 seconds (short, mostly fresh)
// Interest rates: TTL = 5 minutes (rarely changes)

// Fix 2: Active invalidation
// When Transfer Service completes a transfer:
// → Publish event: "balance changed for SAV2001"
// → Gateway receives event → evicts cache key for SAV2001
// → Next request fetches fresh balance

How should the gateway honor HTTP Cache-Control headers?

The backend should control cacheability through standard HTTP headers. The gateway respects Cache-Control: max-age=30 or Cache-Control: no-store.

JAVA

package com.bank.controller;

import org.springframework.http.CacheControl;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.*;
import java.math.BigDecimal;
import java.time.Duration;

@RestController
@RequestMapping("/api/v1/rates")
public class RatesController {

    @GetMapping("/fd")
    public ResponseEntity<BigDecimal> getFdRate() {
        BigDecimal rate = new BigDecimal("6.75");
        return ResponseEntity.ok()
                .cacheControl(CacheControl.maxAge(Duration.ofMinutes(5)))
                .body(rate);
        // Gateway sees: Cache-Control: max-age=300
        // Caches the response for 5 minutes
    }
}

JAVA

package com.bank.controller;

import org.springframework.http.CacheControl;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.*;
import java.math.BigDecimal;

@RestController
@RequestMapping("/api/v1/accounts")
public class AccountController {

    @GetMapping("/{id}/balance")
    public ResponseEntity<BigDecimal> getBalance(@PathVariable String id) {
        BigDecimal balance = new BigDecimal("75000.00");
        return ResponseEntity.ok()
                .cacheControl(CacheControl.noStore())
                .body(balance);
        // Gateway sees: Cache-Control: no-store
        // Never caches this response → always fresh
    }
}

What is the danger of caching sensitive data?

If the gateway caches a response containing Rajesh's balance under a key that does not include his identity, another user hitting the same path receives his data. This is a security breach and a compliance failure.

JAVA

// NEVER cache:
// - Authentication tokens or session data
// - Personal account balances (without user in key)
// - KYC documents or personal information
// - Any response where Cache-Control: no-store

// SAFE to cache (shared, non-personal):
// - Fixed-deposit interest rates (same for everyone)
// - Branch location data
// - Product catalog information
// - Public exchange rates

How do you choose between local and distributed cache?

With multiple gateway replicas, a local cache per replica can be inconsistent. A shared Redis cache is consistent but adds a network hop.

DIAGRAM

// Local cache (per replica):
// Pros: fastest (in-memory, no network)
// Cons: each replica has its own cache → inconsistent
//   Rajesh hits Replica 1: cached balance = 75000
//   Rajesh hits Replica 2: cache miss → fetches fresh = 25000
//   Different answers from the same gateway!

// Distributed cache (Redis):
// Pros: consistent across all replicas
// Cons: adds ~1ms network hop per cache check
//   Rajesh hits any replica → same cached value from Redis

// Decision:
// High-traffic, tolerance for brief inconsistency → local cache
// Consistency matters (banking) → distributed Redis cache

What are common mistakes? (wrong vs correct)

JAVA

// WRONG: caching POST requests
// POST /api/v1/payments/transfer → cached response!
// Next POST returns the cached transfer result
// Customer charged once but sees two transfer confirmations

// CORRECT: only cache GET (safe, idempotent methods)
// POST, PUT, DELETE are never cached at the gateway

// WRONG: caching without user in the key
// GET /api/v1/accounts/balance → key: "/accounts/balance"
// Rajesh's balance leaked to Priya

// CORRECT: include user identity in the key
// GET /api/v1/accounts/balance → key: "/accounts/balance:user=Rajesh"

What should a senior engineer know?

A senior engineer selects what to cache deliberately per endpoint, sets TTLs based on data volatility, and implements active invalidation for data that must be fresh after writes. They monitor cache hit rates and eviction patterns.

DIAGRAM

// Senior caching strategy per endpoint:
// /api/v1/rates/fd       → cache 5m (rarely changes, same for all)
// /api/v1/products       → cache 1h (catalog updates are batched)
// /api/v1/accounts/{id}  → no-store (sensitive, per-user, volatile)
// /api/v1/dashboard/{id} → cache 10s, keyed by user (short TTL, per-user)

// Active invalidation:
// Transfer completes → publish CacheInvalidation("balance:SAV2001")
// Gateway evicts that key → next request fetches fresh balance

Visual Overview

DIAGRAM

  Gateway Caching Flow
  ────────────────────
  Client ──▶ Gateway Cache ──hit──▶ Return cached response (fast)
                  │
                  └──miss──▶ Backend Service
                              │
                              ▼
                  Cache response with TTL
                              │
                              ▼
                  Return fresh response to client

  Stale Data Timeline
  ───────────────────
  T=0:  Balance=75000 → cached (TTL=30s)
  T=5:  Transfer 50000 → actual=25000, cache still says 75000
  T=30: Cache expires → next request returns 25000

  Active Invalidation
  ───────────────────
  T=0:  Balance=75000 → cached
  T=5:  Transfer → invalidation event → cache evicted
  T=6:  Next request → cache miss → fresh balance=25000

Key Takeaway

An API Gateway caches at the HTTP layer by storing responses of safe, idempotent GET requests keyed on the request and served until they expire. For read-heavy, slowly changing data like interest rates, the gateway answers instantly without touching the backend. At National Bank of India, the fixed-deposit rates panel is identical for every customer and changes at most once a day, so caching it at the gateway with a 5-minute TTL handles thousands of requests without a single backend call.

The hard part is the cache key. A key that is too coarse, like just the path, serves one customer's data to another if the response is personalized. A key that is too fine never gets a hit. For public data, key on method, path, and query. For personalized data, include the user identity so Rajesh's balance and Priya's balance are stored under different keys. Never cache authenticated, user-specific responses in a shared cache without the user in the key.

The three problems to always mention are stale data, granularity, and sensitive data. Stale data is the built-in trade-off: anything cached can be out of date for up to its TTL. For volatile data like balances, either keep the TTL very short or implement active invalidation that evicts the cache when the data changes. Caching sensitive data without proper keying is the most dangerous problem because it can leak one customer's data to another, which is both a security breach and a compliance failure.

Remember one sentence: cache safe, shared, slowly changing reads at the gateway with a sensible TTL, and never cache per-user data without the user in the key.

---