Definition
- Cost to offload – JWT signature verification is cryptographic work, and if paired with a network call to fetch keys, the combined cost multiplied by traffic saturates the gateway.
- Local verification – cache the issuer's JWKS (public keys) and verify every token locally with no network call per request.
- Sidecar verification – push the verification into a sidecar proxy next to each service to spread CPU across many instances.
- Dedicated auth tier – a front tier verifies the heavy JWT once, issues a small trusted context token, and inner services check only the cheap token.
- Introspection cache – for opaque tokens, cache the auth-server response per token for a short TTL.
- ECDSA vs RSA – elliptic-curve cryptography achieves equivalent security with smaller keys and generally better verification throughput.
Imagine the security desk at National Bank of India must check a tamper-proof seal on every customer's pass. If checking requires phoning head office each time, the line crawls. Offloading is making the check fast: keep a copy of the seal pattern at the desk so the guard checks on the spot (cached local verification), or have a separate verification office stamp each pass once and give a simple wristband that inner desks accept (dedicated auth tier), or remember the answer for a few minutes (introspection cache). Switching to a seal design that is quicker to inspect is like choosing ECDSA over RSA.
Understanding the Concepts
What cost are you offloading?
JWT verification involves a cryptographic signature check using the issuer's public key. If the gateway fetches the key over the network per request, that network round trip dominates. Even with a cached key, the crypto itself costs CPU at high throughput.
// Per-request JWT verification cost:
// Network fetch of public key: ~5ms (if not cached)
// RSA-2048 signature check: ~0.2ms
// ECDSA-P256 signature check: ~0.1ms
// At 50,000 req/s with uncached RSA:
// 50,000 * 5.2ms = 260,000ms = 260 CPU-seconds/second (impossible!)
// At 50,000 req/s with cached RSA:
// 50,000 * 0.2ms = 10,000ms = 10 CPU-seconds/second (manageable)
// Caching the key alone gives a 26x improvement
How does local verification with cached JWKS work?
A signed JWT is self-contained. The gateway only needs the issuer's public keys, which rarely change. Fetch the JWKS once, cache it, and verify every token locally.
package com.bank.gateway.security;
import org.springframework.security.oauth2.jwt.JwtDecoder;
import org.springframework.security.oauth2.jwt.NimbusJwtDecoder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
@Configuration
public class CachedJwtConfig {
@Bean
public JwtDecoder jwtDecoder() {
// Fetches JWKS once, caches keys in memory
// Automatically refreshes when an unknown keyId appears
return NimbusJwtDecoder.withJwkSetUri(
"https://auth.nbi.example.com/.well-known/jwks.json")
.build();
}
}
// Startup: fetch JWKS from auth server → cache 3 public keys // Request 1: verify JWT with cached key → 0.2ms (no network) // Request 50,000: same cached key → 0.2ms // Key rotation: new keyId appears → auto-refresh JWKS → cache new key // Total network calls: ~1 per key rotation (every few hours)
How does sidecar verification distribute the CPU load?
Putting verification in a sidecar proxy next to each service spreads the crypto work across many instances instead of concentrating it in the gateway.
// Centralized (gateway only): // 50,000 req/s → all JWT verification on 3 gateway replicas // Each replica: ~17,000 verifications/s // Distributed (sidecar per service): // 50,000 req/s → verification in sidecars across 20 service instances // Each sidecar: ~2,500 verifications/s // Gateway freed from crypto work entirely // Sidecar: Envoy, Istio, or a custom proxy // Each sidecar has its own cached JWKS
How does a dedicated auth tier issue a context token?
A front tier verifies the heavy JWT once, then mints a small internal context token that inner services trust cheaply.
// Flow:
// 1. Client sends request with heavy JWT to Auth Tier
// 2. Auth Tier verifies JWT (signature, expiry, scopes)
// 3. Auth Tier issues a lightweight context token:
// {userId: "CUST-1001", roles: ["TRANSFER"], iat: now, exp: now+60s}
// Signed with a symmetric key (HMAC) → cheap to verify
// 4. Forward context token to API Gateway / services
// 5. Services verify context token: HMAC check = ~0.01ms (vs RSA 0.2ms)
// Heavy work (RSA verify): happens ONCE at auth tier
// Cheap work (HMAC verify): happens at every service, but costs almost nothing
// Especially useful when one request fans out to 5+ internal services
How does introspection caching work for opaque tokens?
Opaque tokens (not self-contained JWTs) require a network call to the auth server's introspection endpoint. Cache the result per token.
package com.bank.gateway.security;
import org.springframework.data.redis.core.StringRedisTemplate;
import org.springframework.stereotype.Component;
import java.time.Duration;
@Component
public class IntrospectionCache {
private static final Duration CACHE_TTL = Duration.ofSeconds(30);
private final StringRedisTemplate redis;
private final AuthServerClient authClient;
public IntrospectionCache(StringRedisTemplate redis,
AuthServerClient authClient) {
this.redis = redis;
this.authClient = authClient;
}
public IntrospectionResult introspect(String token) {
// Check cache first
String cached = redis.opsForValue().get("introspect:" + hash(token));
if (cached != null) {
return deserialize(cached); // Cache hit: no network call
}
// Cache miss: call auth server
IntrospectionResult result = authClient.introspect(token);
redis.opsForValue().set(
"introspect:" + hash(token),
serialize(result),
CACHE_TTL);
return result;
}
private String hash(String token) {
// Hash the token for the cache key, never store raw tokens
return null; // SHA-256 hash
}
private String serialize(IntrospectionResult result) { return ""; }
private IntrospectionResult deserialize(String json) { return null; }
}
// Without cache: 50,000 req/s * 5ms introspection call = 250,000ms
// With 30s TTL cache: ~1 call per unique token per 30s
// If 1000 unique tokens active: ~33 calls/s instead of 50,000
// 99.93% reduction in auth-server calls
Why is ECDSA verification cheaper than RSA?
Elliptic-curve cryptography achieves the same security with much smaller keys and signatures, giving better throughput.
// Security equivalence:
// RSA-2048 ≈ ECDSA-P256 (both ~128-bit security)
// Key sizes:
// RSA-2048: public key = 256 bytes, signature = 256 bytes
// ECDSA-P256: public key = 64 bytes, signature = 64 bytes
// 4x smaller keys and signatures
// Verification throughput (approximate, platform-dependent):
// RSA-2048: ~15,000 verifications/second per core
// ECDSA-P256: ~20,000+ verifications/second per core
// Measure on your platform; ECDSA is typically faster or comparable
// JWT header: {"alg": "ES256"} instead of {"alg": "RS256"}
// Smaller tokens, faster verification, same security
What are common mistakes? (wrong vs correct)
// WRONG: network call to auth server per request
IntrospectionResult result = authClient.introspect(token); // 5ms per call
// At 50K req/s → auth server overwhelmed
// CORRECT: cache JWKS locally for self-contained JWTs
JwtDecoder decoder = NimbusJwtDecoder.withJwkSetUri(jwksUrl).build();
// Local verification: 0.2ms per call, no network
// WRONG: never refreshing cached keys
// Key rotation happens → new tokens fail verification!
// CORRECT: refresh when unknown keyId appears
// NimbusJwtDecoder handles this automatically
What should a senior engineer know?
A senior engineer measures verification throughput on their platform, standardizes on ECDSA for new systems, and combines caching with a dedicated auth tier for high-fan-out scenarios.
// Senior offloading strategy: // 1. Cache JWKS locally (eliminates network cost) // 2. ECDSA tokens (faster verification, smaller) // 3. Sidecar verification (distribute CPU across services) // 4. Dedicated auth tier + context token (for fan-out scenarios) // 5. Introspection cache for opaque tokens (30s TTL) // 6. Key rotation: auto-refresh on unknown keyId // 7. Self-contained JWTs cannot be revoked → short TTL + revocation check
Visual Overview
Offloading JWT Verification
───────────────────────────
Technique 1: Cached Local Verification
Auth Server → JWKS (public keys) → Gateway Cache
Every request: local crypto check (no network)
Technique 2: Sidecar Verification
Gateway → forward without verifying → Sidecar Proxy → verify locally
CPU spread across 20 sidecars instead of 3 gateways
Technique 3: Dedicated Auth Tier
Client → Auth Tier (verify heavy JWT once)
→ issue lightweight context token (HMAC)
→ Gateway/Services verify cheaply
Technique 4: Introspection Cache
Client → Gateway → Redis Cache ──hit──▶ return cached result
│
└──miss──▶ Auth Server → cache result (30s TTL)
Key Takeaway
When every request carries a JWT, the cost of verifying it, which is a cryptographic signature check and sometimes a network call, is multiplied by all the traffic and can saturate the gateway's CPU. The most impactful fix is local verification with cached public keys: a signed JWT is self-contained, so the gateway only needs the issuer's JWKS, which it fetches once, caches, and refreshes on key rotation. This alone removes the per-request network cost that was the bottleneck.
At National Bank of India, Pankaj also pushed verification into sidecars next to each service for high-traffic internal calls, spreading the CPU load across many instances instead of concentrating it in the gateway. For scenarios where one request fans out to many internal services, a dedicated auth tier verifies the heavy JWT once and issues a lightweight HMAC context token that inner services check for almost zero cost.
Choosing ECDSA over RSA is a real lever at edge scale. Elliptic-curve cryptography achieves the same security with much smaller keys and signatures, and it gives favorable verification throughput, so a high-throughput gateway can verify more tokens per second with ECDSA.
Remember one sentence: cache the JWKS locally so every JWT verification is a local crypto check with no network call, and choose ECDSA for smaller tokens and faster verification.
---