Back to Scenarios & Design Questions
Q68Question 68 of 68

How do you design an e-commerce checkout workflow with microservices

Answer: Design the checkout workflow as a Saga with a central OrderSagaOrchestrator that coordinates inventory reservation, payment authorization, and notification across independent services, running compensating actions in reverse order when any step fails.

Key Concepts
  • A Saga is a sequence of local transactions across microservices, where each step has a compensating action that undoes it if a later step fails.
  • Orchestration uses a central OrderSagaOrchestrator that calls each service in sequence and manages compensation on failure.
  • The checkout order moves through states: PENDING, INVENTORY_RESERVED, PAYMENT_AUTHORIZED, CONFIRMED (or CANCELLED on failure).
  • Compensating actions reverse completed steps in order: refundPayment() then releaseInventory().
  • Idempotency keys on payment operations prevent duplicate charges when the orchestrator retries after a timeout.
  • The transactional outbox pattern writes business data and events in the same database transaction to prevent lost events.
  • At the senior level, persist the Saga execution log for crash recovery, use optimistic locking for concurrent Sagas, and run dead-Saga detection jobs.

Definition

Design the checkout workflow as a Saga with a central OrderSagaOrchestrator that coordinates inventory reservation, payment authorization, and notification across independent services, running compensating actions in reverse order when any step fails.
  • Saga pattern is a sequence of local transactions in different microservices where each step has a compensating action, used instead of distributed transactions to maintain data consistency across services.
  • Orchestration is a Saga coordination style where a central orchestrator directs each step, making the full workflow visible in one class and simplifying error handling and compensation.
  • Compensating transaction is the reverse operation for a Saga step (releaseInventory, refundPayment) that undoes the step's effect when a later step fails.
  • Transactional outbox is a pattern where the service writes business data and an outbox event record in the same database transaction, and a separate poller publishes the event to Kafka.
  • Idempotency key on payment operations ensures that if the orchestrator retries after a timeout, the payment is authorized only once, preventing duplicate charges.

Think of National Bank of India processing a fixed deposit purchase for Rajesh. The bank clerk (orchestrator) follows a checklist: first, check that the bonds are available (inventory). Second, debit Rajesh's account (payment). Third, print the certificate (notification). If the payment fails at step two, the clerk does not keep the bonds reserved forever. The clerk reverses step one by putting the bonds back on the shelf. Each step in the checklist has a cross-out instruction for what to do if it fails. The clerk follows the checklist in order, and if something goes wrong, follows the cross-out instructions in reverse order.

Understanding the Concepts

What is a Saga pattern?

A Saga is a sequence of local transactions in different microservices, where each step has a compensating action that undoes it if a later step fails. Instead of a single distributed transaction (which requires two-phase commit and locks across databases), a Saga breaks the workflow into steps. If step 3 fails, the Saga runs compensating actions for step 2 and step 1 in reverse order to restore the system to a consistent state.

There are two types of Sagas: choreography (each service listens for events and decides what to do next) and orchestration (a central orchestrator directs each step). Orchestration is easier to understand, debug, and modify, so it is the preferred approach for complex workflows like checkout.

JAVA
package com.bank.saga;

public class SagaConcepts {

    sealed interface SagaStep permits ForwardStep, CompensatingStep {}
    record ForwardStep(int order, String service, String action) implements SagaStep {}
    record CompensatingStep(int order, String service, String action) implements SagaStep {}

    public static void main(String[] args) {
        var forwardSteps = java.util.List.of(
            new ForwardStep(1, "inventory-service", "Reserve 2 units of PROD-001"),
            new ForwardStep(2, "payment-service", "Authorize payment of 150000 INR"),
            new ForwardStep(3, "notification-service", "Send order confirmation email")
        );

        var compensatingSteps = java.util.List.of(
            new CompensatingStep(2, "payment-service", "Refund payment of 150000 INR"),
            new CompensatingStep(1, "inventory-service", "Release 2 units of PROD-001")
        );

        System.out.println("=== Forward Steps (happy path) ===");
        for (var step : forwardSteps) {
            System.out.println("  Step " + step.order() + ": " + step.service() + " -> " + step.action());
        }
        System.out.println();
        System.out.println("=== Compensating Steps (if step 3 fails) ===");
        for (var step : compensatingSteps) {
            System.out.println("  Undo step " + step.order() + ": " + step.service() + " -> " + step.action());
        }
    }
}
TEXT
Expected output:
=== Forward Steps (happy path) ===
  Step 1: inventory-service -> Reserve 2 units of PROD-001
  Step 2: payment-service -> Authorize payment of 150000 INR
  Step 3: notification-service -> Send order confirmation email

=== Compensating Steps (if step 3 fails) ===
  Undo step 2: payment-service -> Refund payment of 150000 INR
  Undo step 1: inventory-service -> Release 2 units of PROD-001

How is orchestration different from choreography?

FeatureChoreographyOrchestration
CoordinationEach service listens for events and decidesCentral orchestrator directs each step
CouplingLoose (services only know about events)Services depend on orchestrator
VisibilityHard to see the full workflowFull workflow visible in orchestrator
DebuggingTrace events across servicesCheck orchestrator state machine
ComplexitySimple for 2-3 stepsBetter for 4+ steps
Error handlingEach service handles its own failuresOrchestrator manages compensation
Adding stepsModify event consumersModify orchestrator only
Circular dependency riskHigh (event chains can loop)Low (linear flow)
JAVA
package com.bank.saga;

public class ChoreographyVsOrchestration {
    public static void main(String[] args) {
        System.out.println("=== Choreography (event-driven) ===");
        System.out.println("  order-service publishes OrderCreated");
        System.out.println("  inventory-service hears OrderCreated, reserves stock, publishes InventoryReserved");
        System.out.println("  payment-service hears InventoryReserved, authorizes payment, publishes PaymentAuthorized");
        System.out.println("  notification-service hears PaymentAuthorized, sends email");
        System.out.println("  Problem: no single place to see the full flow");
        System.out.println("  Problem: if payment fails, who triggers inventory release?");
        System.out.println();

        System.out.println("=== Orchestration (central coordinator) ===");
        System.out.println("  OrderSagaOrchestrator controls the entire flow:");
        System.out.println("  Step 1: call inventory-service.reserveStock()");
        System.out.println("  Step 2: call payment-service.authorizePayment()");
        System.out.println("  Step 3: call notification-service.sendConfirmation()");
        System.out.println("  If step 2 fails: orchestrator calls inventory-service.releaseStock()");
        System.out.println("  All logic in ONE class. Easy to read, debug, and modify.");
    }
}
TEXT
Expected output:
=== Choreography (event-driven) ===
  order-service publishes OrderCreated
  inventory-service hears OrderCreated, reserves stock, publishes InventoryReserved
  payment-service hears InventoryReserved, authorizes payment, publishes PaymentAuthorized
  notification-service hears PaymentAuthorized, sends email
  Problem: no single place to see the full flow
  Problem: if payment fails, who triggers inventory release?

=== Orchestration (central coordinator) ===
  OrderSagaOrchestrator controls the entire flow:
  Step 1: call inventory-service.reserveStock()
  Step 2: call payment-service.authorizePayment()
  Step 3: call notification-service.sendConfirmation()
  If step 2 fails: orchestrator calls inventory-service.releaseStock()
  All logic in ONE class. Easy to read, debug, and modify.

What are the order states in the checkout workflow?

The order moves through a state machine as the Saga progresses. Each forward step transitions the order to the next state. Each compensating step transitions it backward. This state machine makes it easy to understand where any order is in the workflow.

JAVA
package com.bank.saga;

public class OrderStateMachine {

    enum OrderState {
        PENDING,
        INVENTORY_RESERVED,
        PAYMENT_AUTHORIZED,
        CONFIRMED,
        // Failure states
        INVENTORY_FAILED,
        PAYMENT_FAILED,
        PAYMENT_REFUNDED,
        INVENTORY_RELEASED,
        CANCELLED
    }

    record StateTransition(OrderState from, String trigger, OrderState to) {}

    public static void main(String[] args) {
        var happyPath = java.util.List.of(
            new StateTransition(OrderState.PENDING, "reserveInventory() succeeds", OrderState.INVENTORY_RESERVED),
            new StateTransition(OrderState.INVENTORY_RESERVED, "authorizePayment() succeeds", OrderState.PAYMENT_AUTHORIZED),
            new StateTransition(OrderState.PAYMENT_AUTHORIZED, "sendConfirmation() succeeds", OrderState.CONFIRMED)
        );

        var failurePath = java.util.List.of(
            new StateTransition(OrderState.PENDING, "reserveInventory() fails", OrderState.INVENTORY_FAILED),
            new StateTransition(OrderState.INVENTORY_RESERVED, "authorizePayment() fails", OrderState.PAYMENT_FAILED),
            new StateTransition(OrderState.PAYMENT_FAILED, "refundPayment()", OrderState.PAYMENT_REFUNDED),
            new StateTransition(OrderState.PAYMENT_REFUNDED, "releaseInventory()", OrderState.INVENTORY_RELEASED),
            new StateTransition(OrderState.INVENTORY_RELEASED, "markCancelled()", OrderState.CANCELLED)
        );

        System.out.println("=== Happy Path ===");
        happyPath.forEach(t -> System.out.println("  " + t.from() + " -> [" + t.trigger() + "] -> " + t.to()));
        System.out.println();
        System.out.println("=== Failure + Compensation Path ===");
        failurePath.forEach(t -> System.out.println("  " + t.from() + " -> [" + t.trigger() + "] -> " + t.to()));
    }
}
TEXT
Expected output:
=== Happy Path ===
  PENDING -> [reserveInventory() succeeds] -> INVENTORY_RESERVED
  INVENTORY_RESERVED -> [authorizePayment() succeeds] -> PAYMENT_AUTHORIZED
  PAYMENT_AUTHORIZED -> [sendConfirmation() succeeds] -> CONFIRMED

=== Failure + Compensation Path ===
  PENDING -> [reserveInventory() fails] -> INVENTORY_FAILED
  INVENTORY_RESERVED -> [authorizePayment() fails] -> PAYMENT_FAILED
  PAYMENT_FAILED -> [refundPayment()] -> PAYMENT_REFUNDED
  PAYMENT_REFUNDED -> [releaseInventory()] -> INVENTORY_RELEASED
  INVENTORY_RELEASED -> [markCancelled()] -> CANCELLED

What is the transactional outbox pattern?

The transactional outbox pattern solves the dual-write problem: when a service needs to update its database AND publish an event, doing both atomically is impossible with two different systems (database + Kafka). If the database write succeeds but the Kafka publish fails, the event is lost.

The solution: write the event to an outbox table in the same database transaction as the business data. A separate process (outbox poller or CDC connector) reads the outbox table and publishes events to Kafka. Since the business data and the outbox record are in the same transaction, they either both succeed or both roll back.

JAVA
package com.bank.saga;

public class TransactionalOutboxDemo {

    record OutboxEntry(long id, String aggregateType, String aggregateId,
                       String eventType, String payload, boolean published) {}

    public static void main(String[] args) {
        System.out.println("=== Without Outbox (dual-write problem) ===");
        System.out.println("  1. BEGIN TRANSACTION");
        System.out.println("  2. INSERT INTO orders (status='CONFIRMED')  -- SUCCESS");
        System.out.println("  3. COMMIT");
        System.out.println("  4. kafkaTemplate.send('order-events', ...)  -- FAILS (Kafka down)");
        System.out.println("  Result: order is CONFIRMED but event is LOST");
        System.out.println();

        System.out.println("=== With Outbox Pattern ===");
        System.out.println("  1. BEGIN TRANSACTION");
        System.out.println("  2. INSERT INTO orders (status='CONFIRMED')");
        System.out.println("  3. INSERT INTO outbox (event_type='ORDER_CONFIRMED', payload='{...}')");
        System.out.println("  4. COMMIT  (both writes are atomic)");
        System.out.println("  5. Outbox poller reads outbox table every 1 second");
        System.out.println("  6. Poller publishes event to Kafka");
        System.out.println("  7. Poller marks outbox entry as published=true");
        System.out.println("  Result: event is guaranteed to be published eventually");
        System.out.println();

        var entries = java.util.List.of(
            new OutboxEntry(1, "Order", "ORD-501", "ORDER_CONFIRMED",
                "{\"orderId\":\"ORD-501\",\"amount\":150000}", true),
            new OutboxEntry(2, "Order", "ORD-502", "ORDER_CANCELLED",
                "{\"orderId\":\"ORD-502\",\"reason\":\"payment failed\"}", false)
        );

        System.out.println("Outbox table contents:");
        for (var entry : entries) {
            System.out.println("  id=" + entry.id() + ", type=" + entry.eventType()
                + ", aggregateId=" + entry.aggregateId()
                + ", published=" + entry.published());
        }
    }
}
TEXT
Expected output:
=== Without Outbox (dual-write problem) ===
  1. BEGIN TRANSACTION
  2. INSERT INTO orders (status='CONFIRMED')  -- SUCCESS
  3. COMMIT
  4. kafkaTemplate.send('order-events', ...)  -- FAILS (Kafka down)
  Result: order is CONFIRMED but event is LOST

=== With Outbox Pattern ===
  1. BEGIN TRANSACTION
  2. INSERT INTO orders (status='CONFIRMED')
  3. INSERT INTO outbox (event_type='ORDER_CONFIRMED', payload='{...}')
  4. COMMIT  (both writes are atomic)
  5. Outbox poller reads outbox table every 1 second
  6. Poller publishes event to Kafka
  7. Poller marks outbox entry as published=true
  Result: event is guaranteed to be published eventually

Outbox table contents:
  id=1, type=ORDER_CONFIRMED, aggregateId=ORD-501, published=true
  id=2, type=ORDER_CANCELLED, aggregateId=ORD-502, published=false

What is a complete e-commerce checkout project with Saga orchestration?

This project has four Spring Boot services: order-service (Saga orchestrator), inventory-service, payment-service, and notification-service. The order-service coordinates the checkout workflow step by step. If any step fails, the orchestrator runs compensating actions in reverse order.

Project structure

DIAGRAM
checkout-saga/
├── docker-compose.yml
├── init-db.sql
├── order-service/               (Saga orchestrator)
│   ├── pom.xml
│   └── src/main/java/com/bank/order/
│       ├── OrderServiceApp.java
│       ├── controller/CheckoutController.java
│       ├── service/OrderSagaOrchestrator.java
│       ├── client/InventoryClient.java
│       ├── client/PaymentClient.java
│       ├── client/NotificationClient.java
│       ├── model/Order.java
│       ├── model/OutboxEvent.java
│       ├── dto/CheckoutRequest.java
│       ├── dto/CheckoutResponse.java
│       ├── repository/OrderRepository.java
│       ├── repository/OutboxEventRepository.java
│       ├── config/RestClientConfig.java
│       └── exception/SagaFailedException.java
├── inventory-service/
│   ├── pom.xml
│   └── src/main/java/com/bank/inventory/
│       ├── InventoryServiceApp.java
│       ├── controller/InventoryController.java
│       ├── service/InventoryService.java
│       ├── model/Inventory.java
│       ├── dto/ReserveRequest.java
│       ├── dto/ReleaseRequest.java
│       └── repository/InventoryRepository.java
├── payment-service/
│   ├── pom.xml
│   └── src/main/java/com/bank/payment/
│       ├── PaymentServiceApp.java
│       ├── controller/PaymentController.java
│       ├── service/PaymentService.java
│       ├── model/Payment.java
│       ├── model/IdempotencyRecord.java
│       ├── dto/AuthorizeRequest.java
│       ├── dto/RefundRequest.java
│       └── repository/PaymentRepository.java
└── notification-service/
    ├── pom.xml
    └── src/main/java/com/bank/notification/
        ├── NotificationServiceApp.java
        ├── controller/NotificationController.java
        └── service/NotificationService.java

docker-compose.yml

YAML
version: "3.9"
services:
  postgres:
    image: postgres:16
    environment:
      POSTGRES_USER: bank
      POSTGRES_PASSWORD: bank123
      POSTGRES_DB: bank
    ports:
      - "5432:5432"
    volumes:
      - ./init-db.sql:/docker-entrypoint-initdb.d/01-init.sql

  order-service:
    build: ./order-service
    ports:
      - "8080:8080"
    environment:
      SPRING_DATASOURCE_URL: jdbc:postgresql://postgres:5432/bank
      INVENTORY_SERVICE_URL: http://inventory-service:8081
      PAYMENT_SERVICE_URL: http://payment-service:8082
      NOTIFICATION_SERVICE_URL: http://notification-service:8083
    depends_on:
      - postgres

  inventory-service:
    build: ./inventory-service
    ports:
      - "8081:8081"
    environment:
      SPRING_DATASOURCE_URL: jdbc:postgresql://postgres:5432/bank
    depends_on:
      - postgres

  payment-service:
    build: ./payment-service
    ports:
      - "8082:8082"
    environment:
      SPRING_DATASOURCE_URL: jdbc:postgresql://postgres:5432/bank
    depends_on:
      - postgres

  notification-service:
    build: ./notification-service
    ports:
      - "8083:8083"

init-db.sql

SQL
CREATE TABLE IF NOT EXISTS orders (
    id              BIGSERIAL PRIMARY KEY,
    order_id        VARCHAR(50) NOT NULL UNIQUE,
    customer_id     VARCHAR(20) NOT NULL,
    product_id      VARCHAR(20) NOT NULL,
    quantity        INT NOT NULL,
    total_amount    NUMERIC(15,2) NOT NULL,
    status          VARCHAR(30) NOT NULL DEFAULT 'PENDING',
    idempotency_key VARCHAR(50) NOT NULL UNIQUE,
    created_at      TIMESTAMP NOT NULL DEFAULT NOW(),
    updated_at      TIMESTAMP NOT NULL DEFAULT NOW()
);

CREATE TABLE IF NOT EXISTS outbox_events (
    id              BIGSERIAL PRIMARY KEY,
    aggregate_type  VARCHAR(50) NOT NULL,
    aggregate_id    VARCHAR(50) NOT NULL,
    event_type      VARCHAR(50) NOT NULL,
    payload         TEXT NOT NULL,
    published       BOOLEAN NOT NULL DEFAULT FALSE,
    created_at      TIMESTAMP NOT NULL DEFAULT NOW()
);

CREATE TABLE IF NOT EXISTS inventory (
    id              BIGSERIAL PRIMARY KEY,
    product_id      VARCHAR(20) NOT NULL UNIQUE,
    product_name    VARCHAR(100) NOT NULL,
    available_qty   INT NOT NULL DEFAULT 0,
    reserved_qty    INT NOT NULL DEFAULT 0
);

INSERT INTO inventory (product_id, product_name, available_qty) VALUES
    ('PROD-001', 'Gold Savings Bond', 100),
    ('PROD-002', 'Fixed Deposit Certificate', 500),
    ('PROD-003', 'Mutual Fund SIP Unit', 1000);

CREATE TABLE IF NOT EXISTS payments (
    id              BIGSERIAL PRIMARY KEY,
    payment_id      VARCHAR(50) NOT NULL UNIQUE,
    order_id        VARCHAR(50) NOT NULL,
    customer_id     VARCHAR(20) NOT NULL,
    amount          NUMERIC(15,2) NOT NULL,
    status          VARCHAR(20) NOT NULL DEFAULT 'PENDING',
    idempotency_key VARCHAR(50) NOT NULL UNIQUE,
    created_at      TIMESTAMP NOT NULL DEFAULT NOW()
);

CheckoutRequest.java and CheckoutResponse.java

JAVA
package com.bank.order.dto;

import java.math.BigDecimal;

public record CheckoutRequest(
    String customerId,
    String productId,
    int quantity,
    BigDecimal unitPrice,
    String idempotencyKey
) {}
JAVA
package com.bank.order.dto;

import java.math.BigDecimal;

public record CheckoutResponse(
    String orderId,
    String status,
    BigDecimal totalAmount,
    String message
) {}

Order.java

JAVA
package com.bank.order.model;

import jakarta.persistence.Entity;
import jakarta.persistence.Table;
import jakarta.persistence.Id;
import jakarta.persistence.GeneratedValue;
import jakarta.persistence.GenerationType;
import jakarta.persistence.Column;

import java.math.BigDecimal;
import java.time.LocalDateTime;

@Entity
@Table(name = "orders")
public class Order {

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    @Column(name = "order_id", nullable = false, unique = true)
    private String orderId;

    @Column(name = "customer_id", nullable = false)
    private String customerId;

    @Column(name = "product_id", nullable = false)
    private String productId;

    @Column(nullable = false)
    private int quantity;

    @Column(name = "total_amount", nullable = false)
    private BigDecimal totalAmount;

    @Column(nullable = false)
    private String status;

    @Column(name = "idempotency_key", nullable = false, unique = true)
    private String idempotencyKey;

    @Column(name = "created_at")
    private LocalDateTime createdAt = LocalDateTime.now();

    @Column(name = "updated_at")
    private LocalDateTime updatedAt = LocalDateTime.now();

    public Long getId() { return id; }
    public String getOrderId() { return orderId; }
    public void setOrderId(String orderId) { this.orderId = orderId; }
    public String getCustomerId() { return customerId; }
    public void setCustomerId(String customerId) { this.customerId = customerId; }
    public String getProductId() { return productId; }
    public void setProductId(String productId) { this.productId = productId; }
    public int getQuantity() { return quantity; }
    public void setQuantity(int quantity) { this.quantity = quantity; }
    public BigDecimal getTotalAmount() { return totalAmount; }
    public void setTotalAmount(BigDecimal totalAmount) { this.totalAmount = totalAmount; }
    public String getStatus() { return status; }
    public void setStatus(String status) { this.status = status; this.updatedAt = LocalDateTime.now(); }
    public String getIdempotencyKey() { return idempotencyKey; }
    public void setIdempotencyKey(String key) { this.idempotencyKey = key; }
}

OrderSagaOrchestrator.java (the core of the Saga)

JAVA
package com.bank.order.service;

import com.bank.order.client.InventoryClient;
import com.bank.order.client.NotificationClient;
import com.bank.order.client.PaymentClient;
import com.bank.order.dto.CheckoutResponse;
import com.bank.order.exception.SagaFailedException;
import com.bank.order.model.Order;
import com.bank.order.repository.OrderRepository;
import org.springframework.stereotype.Service;
import org.springframework.web.client.RestClientException;

import java.math.BigDecimal;
import java.util.UUID;
import java.util.logging.Logger;

@Service
public class OrderSagaOrchestrator {

    private static final Logger LOG = Logger.getLogger(OrderSagaOrchestrator.class.getName());
    private final OrderRepository orderRepository;
    private final InventoryClient inventoryClient;
    private final PaymentClient paymentClient;
    private final NotificationClient notificationClient;

    public OrderSagaOrchestrator(OrderRepository orderRepository,
                                  InventoryClient inventoryClient,
                                  PaymentClient paymentClient,
                                  NotificationClient notificationClient) {
        this.orderRepository = orderRepository;
        this.inventoryClient = inventoryClient;
        this.paymentClient = paymentClient;
        this.notificationClient = notificationClient;
    }

    public CheckoutResponse executeCheckout(String customerId, String productId,
                                             int quantity, BigDecimal unitPrice,
                                             String idempotencyKey) {
        var orderId = "ORD-" + UUID.randomUUID().toString().substring(0, 8).toUpperCase();
        var totalAmount = unitPrice.multiply(BigDecimal.valueOf(quantity));

        // Create order in PENDING state
        var order = new Order();
        order.setOrderId(orderId);
        order.setCustomerId(customerId);
        order.setProductId(productId);
        order.setQuantity(quantity);
        order.setTotalAmount(totalAmount);
        order.setStatus("PENDING");
        order.setIdempotencyKey(idempotencyKey);
        orderRepository.save(order);
        LOG.info("Saga started: orderId=" + orderId + ", total=" + totalAmount);

        // Step 1: Reserve inventory
        try {
            inventoryClient.reserveStock(productId, quantity, orderId);
            order.setStatus("INVENTORY_RESERVED");
            orderRepository.save(order);
            LOG.info("Step 1 PASSED: inventory reserved for " + orderId);
        } catch (RestClientException ex) {
            order.setStatus("INVENTORY_FAILED");
            orderRepository.save(order);
            LOG.severe("Step 1 FAILED: inventory reservation failed for " + orderId);
            return new CheckoutResponse(orderId, "CANCELLED", totalAmount,
                "Inventory reservation failed: " + ex.getMessage());
        }

        // Step 2: Authorize payment
        try {
            paymentClient.authorizePayment(orderId, customerId, totalAmount, idempotencyKey);
            order.setStatus("PAYMENT_AUTHORIZED");
            orderRepository.save(order);
            LOG.info("Step 2 PASSED: payment authorized for " + orderId);
        } catch (RestClientException ex) {
            LOG.severe("Step 2 FAILED: payment failed for " + orderId + ". Starting compensation.");
            order.setStatus("PAYMENT_FAILED");
            orderRepository.save(order);

            // Compensate Step 1: release inventory
            compensateInventory(order, productId, quantity);
            return new CheckoutResponse(orderId, "CANCELLED", totalAmount,
                "Payment failed, inventory released: " + ex.getMessage());
        }

        // Step 3: Send notification
        try {
            notificationClient.sendConfirmation(orderId, customerId, totalAmount);
            order.setStatus("CONFIRMED");
            orderRepository.save(order);
            LOG.info("Step 3 PASSED: notification sent for " + orderId);
        } catch (RestClientException ex) {
            // Notification failure is non-critical, order is still confirmed
            order.setStatus("CONFIRMED");
            orderRepository.save(order);
            LOG.warning("Step 3 FAILED (non-critical): notification failed for " + orderId);
        }

        LOG.info("Saga completed: orderId=" + orderId + ", status=" + order.getStatus());
        return new CheckoutResponse(orderId, order.getStatus(), totalAmount, "Checkout completed");
    }

    private void compensateInventory(Order order, String productId, int quantity) {
        try {
            inventoryClient.releaseStock(productId, quantity, order.getOrderId());
            order.setStatus("CANCELLED");
            orderRepository.save(order);
            LOG.info("Compensation: inventory released for " + order.getOrderId());
        } catch (RestClientException ex) {
            order.setStatus("COMPENSATION_FAILED");
            orderRepository.save(order);
            LOG.severe("COMPENSATION FAILED: inventory release failed for " + order.getOrderId()
                + ". Manual intervention required!");
        }
    }
}

InventoryClient.java, PaymentClient.java, NotificationClient.java

JAVA
package com.bank.order.client;

import org.springframework.stereotype.Component;
import org.springframework.web.client.RestClient;

import java.util.Map;
import java.util.logging.Logger;

@Component
public class InventoryClient {

    private static final Logger LOG = Logger.getLogger(InventoryClient.class.getName());
    private final RestClient restClient;

    public InventoryClient(RestClient inventoryRestClient) {
        this.restClient = inventoryRestClient;
    }

    public void reserveStock(String productId, int quantity, String orderId) {
        LOG.info("Calling inventory-service: reserve " + quantity + " of " + productId);
        restClient.post()
            .uri("/inventory/reserve")
            .body(Map.of("productId", productId, "quantity", quantity, "orderId", orderId))
            .retrieve()
            .toBodilessEntity();
    }

    public void releaseStock(String productId, int quantity, String orderId) {
        LOG.info("Calling inventory-service: release " + quantity + " of " + productId);
        restClient.post()
            .uri("/inventory/release")
            .body(Map.of("productId", productId, "quantity", quantity, "orderId", orderId))
            .retrieve()
            .toBodilessEntity();
    }
}
JAVA
package com.bank.order.client;

import org.springframework.stereotype.Component;
import org.springframework.web.client.RestClient;

import java.math.BigDecimal;
import java.util.Map;
import java.util.logging.Logger;

@Component
public class PaymentClient {

    private static final Logger LOG = Logger.getLogger(PaymentClient.class.getName());
    private final RestClient restClient;

    public PaymentClient(RestClient paymentRestClient) {
        this.restClient = paymentRestClient;
    }

    public void authorizePayment(String orderId, String customerId,
                                  BigDecimal amount, String idempotencyKey) {
        LOG.info("Calling payment-service: authorize " + amount + " for " + orderId);
        restClient.post()
            .uri("/payments/authorize")
            .body(Map.of("orderId", orderId, "customerId", customerId,
                "amount", amount, "idempotencyKey", idempotencyKey))
            .retrieve()
            .toBodilessEntity();
    }

    public void refundPayment(String orderId, String idempotencyKey) {
        LOG.info("Calling payment-service: refund for " + orderId);
        restClient.post()
            .uri("/payments/refund")
            .body(Map.of("orderId", orderId, "idempotencyKey", idempotencyKey + "-refund"))
            .retrieve()
            .toBodilessEntity();
    }
}
JAVA
package com.bank.order.client;

import org.springframework.stereotype.Component;
import org.springframework.web.client.RestClient;

import java.math.BigDecimal;
import java.util.Map;
import java.util.logging.Logger;

@Component
public class NotificationClient {

    private static final Logger LOG = Logger.getLogger(NotificationClient.class.getName());
    private final RestClient restClient;

    public NotificationClient(RestClient notificationRestClient) {
        this.restClient = notificationRestClient;
    }

    public void sendConfirmation(String orderId, String customerId, BigDecimal amount) {
        LOG.info("Calling notification-service: confirm " + orderId);
        restClient.post()
            .uri("/notifications/order-confirmed")
            .body(Map.of("orderId", orderId, "customerId", customerId, "amount", amount))
            .retrieve()
            .toBodilessEntity();
    }
}

RestClientConfig.java (order-service)

JAVA
package com.bank.order.config;

import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.http.client.SimpleClientHttpRequestFactory;
import org.springframework.web.client.RestClient;

@Configuration
public class RestClientConfig {

    private SimpleClientHttpRequestFactory timedFactory() {
        var factory = new SimpleClientHttpRequestFactory();
        factory.setConnectTimeout(3000);
        factory.setReadTimeout(5000);
        return factory;
    }

    @Bean
    public RestClient inventoryRestClient(
            @Value("\${inventory.service.url:http://localhost:8081}") String url) {
        return RestClient.builder().baseUrl(url).requestFactory(timedFactory()).build();
    }

    @Bean
    public RestClient paymentRestClient(
            @Value("\${payment.service.url:http://localhost:8082}") String url) {
        return RestClient.builder().baseUrl(url).requestFactory(timedFactory()).build();
    }

    @Bean
    public RestClient notificationRestClient(
            @Value("\${notification.service.url:http://localhost:8083}") String url) {
        return RestClient.builder().baseUrl(url).requestFactory(timedFactory()).build();
    }
}

CheckoutController.java

JAVA
package com.bank.order.controller;

import com.bank.order.dto.CheckoutRequest;
import com.bank.order.dto.CheckoutResponse;
import com.bank.order.service.OrderSagaOrchestrator;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/checkout")
public class CheckoutController {

    private final OrderSagaOrchestrator orchestrator;

    public CheckoutController(OrderSagaOrchestrator orchestrator) {
        this.orchestrator = orchestrator;
    }

    @PostMapping
    public ResponseEntity<CheckoutResponse> checkout(@RequestBody CheckoutRequest request) {
        var response = orchestrator.executeCheckout(
            request.customerId(), request.productId(), request.quantity(),
            request.unitPrice(), request.idempotencyKey());
        return ResponseEntity.ok(response);
    }
}

InventoryService.java

JAVA
package com.bank.inventory.service;

import com.bank.inventory.model.Inventory;
import com.bank.inventory.repository.InventoryRepository;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

import java.util.logging.Logger;

@Service
public class InventoryService {

    private static final Logger LOG = Logger.getLogger(InventoryService.class.getName());
    private final InventoryRepository repository;

    public InventoryService(InventoryRepository repository) {
        this.repository = repository;
    }

    @Transactional
    public void reserveStock(String productId, int quantity, String orderId) {
        var inventory = repository.findByProductId(productId)
            .orElseThrow(() -> new RuntimeException("Product not found: " + productId));
        if (inventory.getAvailableQty() < quantity) {
            throw new RuntimeException("Insufficient stock for " + productId
                + ": available=" + inventory.getAvailableQty() + ", requested=" + quantity);
        }
        inventory.setAvailableQty(inventory.getAvailableQty() - quantity);
        inventory.setReservedQty(inventory.getReservedQty() + quantity);
        repository.save(inventory);
        LOG.info("Stock reserved: product=" + productId + ", qty=" + quantity + ", orderId=" + orderId);
    }

    @Transactional
    public void releaseStock(String productId, int quantity, String orderId) {
        var inventory = repository.findByProductId(productId)
            .orElseThrow(() -> new RuntimeException("Product not found: " + productId));
        inventory.setAvailableQty(inventory.getAvailableQty() + quantity);
        inventory.setReservedQty(inventory.getReservedQty() - quantity);
        repository.save(inventory);
        LOG.info("Stock released: product=" + productId + ", qty=" + quantity + ", orderId=" + orderId);
    }
}

PaymentService.java (with idempotency)

JAVA
package com.bank.payment.service;

import com.bank.payment.model.Payment;
import com.bank.payment.repository.PaymentRepository;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

import java.math.BigDecimal;
import java.util.UUID;
import java.util.logging.Logger;

@Service
public class PaymentService {

    private static final Logger LOG = Logger.getLogger(PaymentService.class.getName());
    private final PaymentRepository repository;

    public PaymentService(PaymentRepository repository) {
        this.repository = repository;
    }

    @Transactional
    public void authorizePayment(String orderId, String customerId,
                                  BigDecimal amount, String idempotencyKey) {
        // Idempotency check
        if (repository.existsByIdempotencyKey(idempotencyKey)) {
            LOG.info("Duplicate payment authorization: key=" + idempotencyKey);
            return;
        }
        var payment = new Payment();
        payment.setPaymentId("PAY-" + UUID.randomUUID().toString().substring(0, 8).toUpperCase());
        payment.setOrderId(orderId);
        payment.setCustomerId(customerId);
        payment.setAmount(amount);
        payment.setStatus("AUTHORIZED");
        payment.setIdempotencyKey(idempotencyKey);
        repository.save(payment);
        LOG.info("Payment authorized: orderId=" + orderId + ", amount=" + amount);
    }

    @Transactional
    public void refundPayment(String orderId, String idempotencyKey) {
        if (repository.existsByIdempotencyKey(idempotencyKey)) {
            LOG.info("Duplicate refund: key=" + idempotencyKey);
            return;
        }
        var original = repository.findByOrderIdAndStatus(orderId, "AUTHORIZED")
            .orElseThrow(() -> new RuntimeException("No authorized payment for " + orderId));
        original.setStatus("REFUNDED");
        repository.save(original);
        LOG.info("Payment refunded: orderId=" + orderId);
    }
}

How to run and test

BASH
# Step 1: Start all infrastructure
docker-compose up -d

# Step 2: Build all services
for svc in order-service inventory-service payment-service notification-service; do
  cd $svc && mvn clean package -DskipTests && cd ..
done

# Step 3: Start all services
java -jar order-service/target/order-service-1.0.0.jar &
java -jar inventory-service/target/inventory-service-1.0.0.jar &
java -jar payment-service/target/payment-service-1.0.0.jar &
java -jar notification-service/target/notification-service-1.0.0.jar &

# Step 4: Successful checkout (Rajesh buys 2 Gold Savings Bonds at 75000 each)
curl -s -X POST http://localhost:8080/checkout \
  -H "Content-Type: application/json" \
  -d '{"customerId":"SAV2001","productId":"PROD-001","quantity":2,"unitPrice":75000,"idempotencyKey":"CHK-001"}'

# Step 5: Failed checkout (insufficient stock, product has only 100 units)
curl -s -X POST http://localhost:8080/checkout \
  -H "Content-Type: application/json" \
  -d '{"customerId":"SAV2002","productId":"PROD-001","quantity":200,"unitPrice":75000,"idempotencyKey":"CHK-002"}'
TEXT
Expected output (successful checkout):
{
  "orderId": "ORD-A1B2C3D4",
  "status": "CONFIRMED",
  "totalAmount": 150000.00,
  "message": "Checkout completed"
}

Logs (order-service):
  Saga started: orderId=ORD-A1B2C3D4, total=150000.00
  Step 1 PASSED: inventory reserved for ORD-A1B2C3D4
  Step 2 PASSED: payment authorized for ORD-A1B2C3D4
  Step 3 PASSED: notification sent for ORD-A1B2C3D4
  Saga completed: orderId=ORD-A1B2C3D4, status=CONFIRMED

Expected output (failed checkout, insufficient stock):
{
  "orderId": "ORD-E5F6G7H8",
  "status": "CANCELLED",
  "totalAmount": 15000000.00,
  "message": "Inventory reservation failed: Insufficient stock..."
}
BASH
# Step 6: Simulate payment failure (stop payment-service and try checkout)
kill $(lsof -ti:8082)  # Stop payment-service

curl -s -X POST http://localhost:8080/checkout \
  -H "Content-Type: application/json" \
  -d '{"customerId":"SAV2001","productId":"PROD-002","quantity":1,"unitPrice":50000,"idempotencyKey":"CHK-003"}'
TEXT
Expected output (payment-service is down, compensation runs):
{
  "orderId": "ORD-I9J0K1L2",
  "status": "CANCELLED",
  "totalAmount": 50000.00,
  "message": "Payment failed, inventory released: I/O error..."
}

Logs (order-service):
  Saga started: orderId=ORD-I9J0K1L2, total=50000.00
  Step 1 PASSED: inventory reserved for ORD-I9J0K1L2
  Step 2 FAILED: payment failed for ORD-I9J0K1L2. Starting compensation.
  Compensation: inventory released for ORD-I9J0K1L2

What are common mistakes in Saga implementations?

Wrong: no compensating actions (money lost on failure)

JAVA
package com.bank.saga.wrong;

import java.math.BigDecimal;

public class WrongNoCompensation {
    static int inventoryReserved = 0;
    static BigDecimal paymentAuthorized = BigDecimal.ZERO;

    public static void main(String[] args) {
        System.out.println("WRONG: no compensating actions");
        System.out.println();

        // Step 1: Reserve inventory
        inventoryReserved = 2;
        System.out.println("Step 1: Reserved 2 units. inventoryReserved=" + inventoryReserved);

        // Step 2: Authorize payment
        paymentAuthorized = new BigDecimal("150000");
        System.out.println("Step 2: Payment authorized. amount=" + paymentAuthorized);

        // Step 3: Notification FAILS
        System.out.println("Step 3: Notification FAILED (email server down)");
        System.out.println();

        // No compensation!
        System.out.println("Result: inventory is reserved but customer was not notified.");
        System.out.println("  If the order is cancelled, inventory stays locked forever.");
        System.out.println("  If payment was authorized, money is held indefinitely.");
        System.out.println("  NO rollback happened. System is in inconsistent state.");
    }
}
TEXT
Expected output:
WRONG: no compensating actions

Step 1: Reserved 2 units. inventoryReserved=2
Step 2: Payment authorized. amount=150000
Step 3: Notification FAILED (email server down)

Result: inventory is reserved but customer was not notified.
  If the order is cancelled, inventory stays locked forever.
  If payment was authorized, money is held indefinitely.
  NO rollback happened. System is in inconsistent state.

Correct: Saga orchestrator with compensating actions

JAVA
package com.bank.saga.correct;

import java.math.BigDecimal;

public class CorrectWithCompensation {
    static int inventoryReserved = 0;
    static BigDecimal paymentAuthorized = BigDecimal.ZERO;

    public static void main(String[] args) {
        System.out.println("CORRECT: Saga with compensating actions");
        System.out.println();

        // Step 1: Reserve inventory
        inventoryReserved = 2;
        System.out.println("Step 1: Reserved 2 units. inventoryReserved=" + inventoryReserved);

        // Step 2: Authorize payment FAILS
        System.out.println("Step 2: Payment FAILED (payment gateway timeout)");
        System.out.println();

        // Compensate Step 1: release inventory
        System.out.println("Compensating Step 1: releasing 2 units...");
        inventoryReserved = 0;
        System.out.println("  inventoryReserved=" + inventoryReserved);
        System.out.println();

        System.out.println("Result: inventory released, order cancelled.");
        System.out.println("  System is back to consistent state.");
        System.out.println("  Customer can try again.");
    }
}
TEXT
Expected output:
CORRECT: Saga with compensating actions

Step 1: Reserved 2 units. inventoryReserved=2
Step 2: Payment FAILED (payment gateway timeout)

Compensating Step 1: releasing 2 units...
  inventoryReserved=0

Result: inventory released, order cancelled.
  System is back to consistent state.
  Customer can try again.

What should a senior or principal engineer know?

  • At the senior level:
  • Saga execution log: persist every step (action + result) in a saga_execution_log table. This lets you resume a Saga after a crash. On restart, the orchestrator reads the log and continues from the last successful step.
  • Timeout per step: each step should have its own timeout. If inventory-service takes more than 5 seconds, fail the step and start compensation. Do not wait indefinitely.
  • Non-critical steps: notification is a non-critical step. If it fails, the order should still be confirmed. Design your Saga to distinguish between critical and non-critical steps.
  • At the principal level:
  • Saga concurrency: two Sagas might try to reserve the same inventory simultaneously. Use optimistic locking (version column) or pessimistic locking (SELECT FOR UPDATE) in inventory-service to prevent overselling.
  • Compensation idempotency: compensating actions must be idempotent too. If releaseStock() is called twice for the same order, it should only release once.
  • Dead Saga detection: if the orchestrator crashes mid-Saga, the order is stuck in an intermediate state. Run a periodic job that detects orders in non-terminal states older than 5 minutes and either resumes or cancels them.
  • Distributed tracing: use Micrometer Tracing with Zipkin or Jaeger to trace the entire Saga across all four services.
JAVA
package com.bank.saga;

public class SeniorSagaKnowledge {
    public static void main(String[] args) {
        System.out.println("Senior/Principal Saga checklist:");
        System.out.println("  1. Persist saga execution log for crash recovery");
        System.out.println("  2. Set timeouts on each saga step");
        System.out.println("  3. Distinguish critical vs non-critical steps");
        System.out.println("  4. Make compensating actions idempotent");
        System.out.println("  5. Use idempotency keys on payment operations");
        System.out.println("  6. Handle concurrent sagas with optimistic locking");
        System.out.println("  7. Dead saga detection job (find stuck orders)");
        System.out.println("  8. Outbox pattern for reliable event publishing");
        System.out.println("  9. Distributed tracing across all saga participants");
        System.out.println("  10. Alert on COMPENSATION_FAILED status (needs manual fix)");
    }
}
TEXT
Expected output:
Senior/Principal Saga checklist:
  1. Persist saga execution log for crash recovery
  2. Set timeouts on each saga step
  3. Distinguish critical vs non-critical steps
  4. Make compensating actions idempotent
  5. Use idempotency keys on payment operations
  6. Handle concurrent sagas with optimistic locking
  7. Dead saga detection job (find stuck orders)
  8. Outbox pattern for reliable event publishing
  9. Distributed tracing across all saga participants
  10. Alert on COMPENSATION_FAILED status (needs manual fix)

What are common interview follow-up questions?

  • What happens if a compensating action fails? (This is the hardest problem. Log the failure as COMPENSATION_FAILED and alert the operations team. A human must investigate and fix the inconsistency manually. You can also retry the compensation with exponential backoff.)
  • Can you mix choreography and orchestration? (Yes. Use orchestration for the main checkout workflow and choreography for downstream events like analytics and audit logging. The orchestrator handles the critical path; choreography handles fire-and-forget events.)
  • How do you test a Saga end-to-end? (Use Testcontainers to start all four services and PostgreSQL in Docker. Run the checkout flow with a test client. Verify order state, inventory counts, and payment records. Simulate failures by stopping individual containers mid-Saga.)
  • What is the difference between Saga and two-phase commit (2PC)? (2PC locks resources across all participants until the coordinator decides to commit or abort. It provides atomicity but creates tight coupling and is slow. Saga uses eventual consistency with compensating actions. It is faster and more resilient but the system is temporarily inconsistent during execution.)
  • How does the Saga pattern work with event sourcing? (Each Saga step produces an event that is appended to the event store. The Saga state is rebuilt by replaying events. This gives you a complete audit trail and enables time-travel debugging.)

Visual Overview

DIAGRAM
 E-Commerce Checkout Saga (Orchestration)
 ==========================================

                       ┌──────────────────────┐
                       │   CheckoutController  │
                       │   POST /checkout      │
                       └──────────┬───────────┘
                                  │
                                  ▼
                       ┌──────────────────────┐
                       │ OrderSagaOrchestrator │
                       │   (coordinates all)   │
                       └──────────┬───────────┘
                                  │
          ┌───────────────────────┼───────────────────────┐
          │                       │                       │
          ▼                       ▼                       ▼
 ┌─────────────────┐   ┌─────────────────┐   ┌───────────────────┐
 │ inventory-service│   │ payment-service  │   │notification-service│
 │ (port 8081)      │   │ (port 8082)      │   │ (port 8083)        │
 │                  │   │                  │   │                    │
 │ reserveStock()   │   │ authorizePayment()│  │ sendConfirmation() │
 │ releaseStock()   │   │ refundPayment()  │   │                    │
 └────────┬────────┘   └────────┬─────────┘   └────────────────────┘
          │                     │
          ▼                     ▼
 ┌─────────────────┐   ┌─────────────────┐
 │ inventory table  │   │ payments table   │
 └─────────────────┘   └─────────────────┘

 Happy Path:
   PENDING -> INVENTORY_RESERVED -> PAYMENT_AUTHORIZED -> CONFIRMED

 Failure at Step 2 (payment fails):
   PENDING -> INVENTORY_RESERVED -> PAYMENT_FAILED
   Compensation: releaseStock()
   PAYMENT_FAILED -> CANCELLED

 Failure at Step 1 (no stock):
   PENDING -> INVENTORY_FAILED -> CANCELLED (no compensation needed)

Key Takeaway

The Saga pattern breaks a distributed transaction into a sequence of local transactions, each with a compensating action that undoes it if a later step fails. In an e-commerce checkout, the Saga coordinates inventory reservation, payment authorization, and notification delivery across independent microservices. If the payment fails after inventory is reserved, the Saga runs the compensating action to release the inventory and cancel the order.

Orchestration is the preferred approach for complex workflows because the entire flow is visible in one class: OrderSagaOrchestrator. The orchestrator calls each service in sequence, updates the order state after each step, and runs compensating actions in reverse order on failure. Choreography, where each service listens for events and decides what to do next, works for simple flows but becomes hard to trace and debug when there are four or more participants.

In our National Bank of India checkout project, Rajesh bought 2 Gold Savings Bonds at 75,000 rupees each. The orchestrator first called inventory-service to reserve 2 units of PROD-001. Then it called payment-service to authorize 150,000 rupees. Finally, it called notification-service to send a confirmation. Each step updated the order status (PENDING, INVENTORY_RESERVED, PAYMENT_AUTHORIZED, CONFIRMED). When we simulated a payment failure by stopping payment-service, the orchestrator detected the failure, released the inventory, and set the order status to CANCELLED. The system returned to a consistent state.

The transactional outbox pattern complements the Saga by solving the dual-write problem: instead of updating the database and publishing a Kafka event separately (where one can fail), you write both the business data and the outbox record in the same database transaction. A separate poller publishes the event to Kafka later. This guarantees that every state change is eventually published.

At the senior level, make every compensating action idempotent, persist the Saga execution log for crash recovery, and distinguish critical steps (inventory, payment) from non-critical steps (notification). At the principal level, handle concurrent Sagas with optimistic locking, run dead-Saga detection jobs, and use distributed tracing to follow the checkout flow across all four services.

Remember one sentence: a Saga orchestrator coordinates a sequence of local transactions across microservices, updates order state after each step, and runs compensating actions in reverse order when any step fails.

---

Share
Use ← → arrow keys to navigate