What an API Gateway is, its functions, and when to use it | Microservices

Definition

The API Gateway is the single front door that routes, secures, limits, aggregates, caches, and protects, so clients talk to one address and services stop repeating cross-cutting work.

API Gateway – a single entry point that sits in front of backend microservices and handles cross-cutting concerns on their behalf.
Routing – maps incoming request paths to the correct backend service using predicates and load-balanced URIs.
Authentication – validates JWT tokens once at the edge and forwards a trusted identity context downstream.
Rate limiting – counts requests per client and rejects excess traffic with HTTP 429, backed by Redis.
Aggregation – calls multiple services and merges responses into one payload using parallel WebClient + Mono.zip.
Fault tolerance – wraps downstream calls in circuit breakers, timeouts, and fallbacks.
Thin gateway – handles cross-cutting concerns only, never business logic.

Think of the main entrance of National Bank of India. Customers like Rajesh and Priya do not wander into the back offices looking for the loans desk, the accounts desk, and the cards desk. They enter through one front door where a security guard checks their ID, a receptionist directs them to the right department, and a token system limits crowding. The front door does not do the actual banking work; it just controls and directs the flow.

Understanding the Concepts

What is the routing function?

Routing is the core function. The gateway maps an incoming path to a backend service using predicates.

JAVA

package com.bank.gateway;

import org.springframework.cloud.gateway.route.RouteLocator;
import org.springframework.cloud.gateway.route.builder.RouteLocatorBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class RoutingConfig {

    @Bean
    public RouteLocator bankRoutes(RouteLocatorBuilder builder) {
        return builder.routes()
                .route("accounts", r -> r
                        .path("/api/v1/accounts/**")
                        .uri("lb://account-service"))
                .route("payments", r -> r
                        .path("/api/v1/payments/**")
                        .uri("lb://payment-service"))
                .build();
    }
}

DIAGRAM

// /api/v1/accounts/SAV2001/balance → lb://account-service
// /api/v1/payments/transfer → lb://payment-service
// "lb://" means resolve via load balancer + service discovery

How does authentication work at the gateway?

The gateway validates the JWT once at the edge and forwards a trusted identity context downstream. Services do not reimplement token checking.

JAVA

package com.bank.gateway.filter;

import org.springframework.cloud.gateway.filter.GatewayFilterChain;
import org.springframework.cloud.gateway.filter.GlobalFilter;
import org.springframework.http.HttpStatus;
import org.springframework.stereotype.Component;
import org.springframework.web.server.ServerWebExchange;
import reactor.core.publisher.Mono;

@Component
public class JwtAuthFilter implements GlobalFilter {

    private final JwtDecoder jwtDecoder;

    public JwtAuthFilter(JwtDecoder jwtDecoder) {
        this.jwtDecoder = jwtDecoder;
    }

    @Override
    public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
        String token = exchange.getRequest().getHeaders().getFirst("Authorization");
        if (token == null || !token.startsWith("Bearer ")) {
            exchange.getResponse().setStatusCode(HttpStatus.UNAUTHORIZED);
            return exchange.getResponse().setComplete();
        }

        try {
            var claims = jwtDecoder.decode(token.substring(7));
            // Forward trusted identity downstream
            exchange.getRequest().mutate()
                    .header("X-User-Id", claims.getSubject())
                    .header("X-User-Roles", String.join(",", claims.getRoles()));
            return chain.filter(exchange);
        } catch (Exception e) {
            exchange.getResponse().setStatusCode(HttpStatus.UNAUTHORIZED);
            return exchange.getResponse().setComplete();
        }
    }
}

DIAGRAM

// Request with Bearer token → gateway validates signature and expiry
// Valid: adds X-User-Id, X-User-Roles headers → forwards to service
// Invalid: returns 401 Unauthorized → service never sees the request

How does rate limiting protect services?

Rate limiting counts requests per client and rejects excess traffic with HTTP 429. Spring Cloud Gateway uses a RequestRateLimiter filter backed by Redis.

YAML

# application.yml
spring:
  cloud:
    gateway:
      routes:
        - id: payments
          uri: lb://payment-service
          predicates:
            - Path=/api/v1/payments/**
          filters:
            - name: RequestRateLimiter
              args:
                redis-rate-limiter.replenishRate: 100  # 100 tokens/second
                redis-rate-limiter.burstCapacity: 200   # burst up to 200
                key-resolver: "#{@userKeyResolver}"

JAVA

package com.bank.gateway;

import org.springframework.cloud.gateway.filter.ratelimit.KeyResolver;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import reactor.core.publisher.Mono;

@Configuration
public class RateLimitConfig {

    @Bean
    public KeyResolver userKeyResolver() {
        // Rate limit per user (from JWT subject)
        return exchange -> Mono.justOrEmpty(
                exchange.getRequest().getHeaders().getFirst("X-User-Id"));
    }
}

TEXT

// User CUST-1001 sends 150 requests in 1 second:
// First 100: processed normally (replenish rate)
// Next 100: burst capacity allows up to 200 total
// Beyond 200: HTTP 429 Too Many Requests

How does aggregation work?

Aggregation calls multiple services and merges responses into one payload. The client makes one call instead of many.

JAVA

package com.bank.gateway;

import org.springframework.web.bind.annotation.*;
import org.springframework.web.reactive.function.client.WebClient;
import reactor.core.publisher.Mono;
import java.time.Duration;
import java.util.Map;

@RestController
public class DashboardAggregator {

    private final WebClient webClient;

    public DashboardAggregator(WebClient.Builder builder) {
        this.webClient = builder.build();
    }

    @GetMapping("/api/v1/dashboard/{customerId}")
    public Mono<Map<String, Object>> dashboard(@PathVariable String customerId) {
        Mono<Object> account = webClient.get()
                .uri("http://account-service/api/v1/accounts/{id}", customerId)
                .retrieve().bodyToMono(Object.class)
                .timeout(Duration.ofMillis(300))
                .onErrorReturn(Map.of("status", "unavailable"));

        Mono<Object> payments = webClient.get()
                .uri("http://payment-service/api/v1/payments/{id}/recent", customerId)
                .retrieve().bodyToMono(Object.class)
                .timeout(Duration.ofMillis(300))
                .onErrorReturn(Map.of("status", "unavailable"));

        return Mono.zip(account, payments)
                .map(t -> Map.of("account", t.getT1(), "payments", t.getT2()));
    }
}

TEXT

// Mobile app: 1 call to /api/v1/dashboard/CUST-1001
// Gateway: parallel calls to account-service + payment-service
// Response: merged {account: {...}, payments: [...]}
// Time: max(account latency, payment latency) = ~100ms

How does fault tolerance protect the gateway?

Circuit breakers wrap downstream calls. When a service is down, the gateway fails fast with a fallback instead of hanging.

JAVA

// Route with circuit breaker and fallback:
// .filters(f -> f.circuitBreaker(cb -> cb
//         .setName("paymentCB")
//         .setFallbackUri("forward:/fallback/payments")))

// Payment Service healthy: request goes through normally
// Payment Service down: 5 failures → circuit OPEN → fallback in <1ms
// After 30s: HALF_OPEN → test call → success → CLOSED

When should you use an API Gateway?

Use it when you have multiple microservices and multiple client types. It removes duplication of auth, rate limiting, and routing, and decouples clients from the internal layout.

TEXT

// Without gateway:
// Mobile app knows: account-service:8081, payment-service:8082, card-service:8083
// Each service implements: JWT validation, rate limiting, CORS, logging
// Client makes 3 calls for the dashboard

// With gateway:
// Mobile app knows: api-gateway:443 (one address)
// Gateway handles: JWT, rate limiting, CORS, logging (once)
// Client makes 1 call for the dashboard (gateway aggregates)

What are common mistakes? (wrong vs correct)

The most common mistake is putting business logic in the gateway.

JAVA

// WRONG: business logic in the gateway
@PostMapping("/api/v1/transfers")
public Mono<Object> transfer(@RequestBody TransferRequest req) {
    // Fee calculation in the gateway!
    java.math.BigDecimal fee = req.amount()
            .multiply(new java.math.BigDecimal("0.01"));
    // This is business logic that belongs in Payment Service
}

// CORRECT: gateway only routes
// Route: /api/v1/transfers → lb://payment-service
// Payment Service handles: fee calculation, balance check, transfer logic

What should a senior engineer know?

A senior engineer keeps the gateway thin and horizontally scalable. Multiple gateway replicas sit behind a load balancer. For different client types, they use Backend-for-Frontend (BFF) with a tailored gateway per client.

DIAGRAM

// Senior architecture:
//                    ┌─ Mobile BFF Gateway (lean payload)
// Load Balancer ────┤
//                    └─ Web BFF Gateway (rich payload)
//                         │
//              ┌──────────┼──────────┐
//          Account     Payment      Card
//          Service     Service     Service

Visual Overview

DIAGRAM

  API Gateway Functions
  ─────────────────────
  Client Request
       │
       ▼
  ┌────────────────────────────────────────┐
  │              API Gateway                │
  │                                         │
  │  1. Auth Filter    → validate JWT       │
  │  2. Rate Limiter   → token bucket/Redis │
  │  3. Router         → path → service     │
  │  4. Circuit Breaker→ fail fast if down  │
  │  5. Load Balancer  → pick healthy inst  │
  │  6. Aggregator     → merge responses    │
  │  7. Cache          → serve cached GETs  │
  │  8. Tracing        → stamp traceId      │
  └──────┬──────────┬──────────┬────────────┘
         │          │          │
         ▼          ▼          ▼
  Account Svc  Payment Svc  Card Svc

Key Takeaway

An API Gateway is a single entry point that sits in front of all backend microservices and handles the cross-cutting concerns on their behalf. Clients send every request to the gateway, and it takes care of routing, authentication, rate limiting, load balancing, response aggregation, caching, logging, tracing, and fault tolerance. The whole idea is to centralize what would otherwise be duplicated in every service and every client, and to decouple clients from the internal service layout.

At National Bank of India, the mobile app, web app, and partner integrations all talk to one gateway address. The gateway validates the JWT once, routes by path, rate-limits per client via Redis, and load-balances across service instances. Aggregation simplified the mobile dashboard: instead of the app making three calls, it makes one, and the gateway merges account and payment data into a single response. When the card service had a brief outage, the circuit breaker tripped and the dashboard showed "unavailable" for cards while account and payment data still rendered normally.

The discipline that makes an API Gateway work is keeping it thin. It handles routing, security, aggregation, and resilience, but never business rules. The moment you push domain logic into the gateway, it becomes a new monolith and recreates the coupling microservices were meant to remove.

Remember one sentence: the API Gateway is the single thin front door that routes, secures, aggregates, and protects, so clients talk to one address and business logic stays in the services.

---