Definition
- Saga pattern is a sequence of local transactions in different microservices where each step has a compensating action, used instead of distributed transactions to maintain data consistency across services.
- Orchestration is a Saga coordination style where a central orchestrator directs each step, making the full workflow visible in one class and simplifying error handling and compensation.
- Compensating transaction is the reverse operation for a Saga step (releaseInventory, refundPayment) that undoes the step's effect when a later step fails.
- Transactional outbox is a pattern where the service writes business data and an outbox event record in the same database transaction, and a separate poller publishes the event to Kafka.
- Idempotency key on payment operations ensures that if the orchestrator retries after a timeout, the payment is authorized only once, preventing duplicate charges.
Think of National Bank of India processing a fixed deposit purchase for Rajesh. The bank clerk (orchestrator) follows a checklist: first, check that the bonds are available (inventory). Second, debit Rajesh's account (payment). Third, print the certificate (notification). If the payment fails at step two, the clerk does not keep the bonds reserved forever. The clerk reverses step one by putting the bonds back on the shelf. Each step in the checklist has a cross-out instruction for what to do if it fails. The clerk follows the checklist in order, and if something goes wrong, follows the cross-out instructions in reverse order.
Understanding the Concepts
What is a Saga pattern?
A Saga is a sequence of local transactions in different microservices, where each step has a compensating action that undoes it if a later step fails. Instead of a single distributed transaction (which requires two-phase commit and locks across databases), a Saga breaks the workflow into steps. If step 3 fails, the Saga runs compensating actions for step 2 and step 1 in reverse order to restore the system to a consistent state.
There are two types of Sagas: choreography (each service listens for events and decides what to do next) and orchestration (a central orchestrator directs each step). Orchestration is easier to understand, debug, and modify, so it is the preferred approach for complex workflows like checkout.
package com.bank.saga;
public class SagaConcepts {
sealed interface SagaStep permits ForwardStep, CompensatingStep {}
record ForwardStep(int order, String service, String action) implements SagaStep {}
record CompensatingStep(int order, String service, String action) implements SagaStep {}
public static void main(String[] args) {
var forwardSteps = java.util.List.of(
new ForwardStep(1, "inventory-service", "Reserve 2 units of PROD-001"),
new ForwardStep(2, "payment-service", "Authorize payment of 150000 INR"),
new ForwardStep(3, "notification-service", "Send order confirmation email")
);
var compensatingSteps = java.util.List.of(
new CompensatingStep(2, "payment-service", "Refund payment of 150000 INR"),
new CompensatingStep(1, "inventory-service", "Release 2 units of PROD-001")
);
System.out.println("=== Forward Steps (happy path) ===");
for (var step : forwardSteps) {
System.out.println(" Step " + step.order() + ": " + step.service() + " -> " + step.action());
}
System.out.println();
System.out.println("=== Compensating Steps (if step 3 fails) ===");
for (var step : compensatingSteps) {
System.out.println(" Undo step " + step.order() + ": " + step.service() + " -> " + step.action());
}
}
}
Expected output:
=== Forward Steps (happy path) ===
Step 1: inventory-service -> Reserve 2 units of PROD-001
Step 2: payment-service -> Authorize payment of 150000 INR
Step 3: notification-service -> Send order confirmation email
=== Compensating Steps (if step 3 fails) ===
Undo step 2: payment-service -> Refund payment of 150000 INR
Undo step 1: inventory-service -> Release 2 units of PROD-001
How is orchestration different from choreography?
| Feature | Choreography | Orchestration |
|---|---|---|
| Coordination | Each service listens for events and decides | Central orchestrator directs each step |
| Coupling | Loose (services only know about events) | Services depend on orchestrator |
| Visibility | Hard to see the full workflow | Full workflow visible in orchestrator |
| Debugging | Trace events across services | Check orchestrator state machine |
| Complexity | Simple for 2-3 steps | Better for 4+ steps |
| Error handling | Each service handles its own failures | Orchestrator manages compensation |
| Adding steps | Modify event consumers | Modify orchestrator only |
| Circular dependency risk | High (event chains can loop) | Low (linear flow) |
package com.bank.saga;
public class ChoreographyVsOrchestration {
public static void main(String[] args) {
System.out.println("=== Choreography (event-driven) ===");
System.out.println(" order-service publishes OrderCreated");
System.out.println(" inventory-service hears OrderCreated, reserves stock, publishes InventoryReserved");
System.out.println(" payment-service hears InventoryReserved, authorizes payment, publishes PaymentAuthorized");
System.out.println(" notification-service hears PaymentAuthorized, sends email");
System.out.println(" Problem: no single place to see the full flow");
System.out.println(" Problem: if payment fails, who triggers inventory release?");
System.out.println();
System.out.println("=== Orchestration (central coordinator) ===");
System.out.println(" OrderSagaOrchestrator controls the entire flow:");
System.out.println(" Step 1: call inventory-service.reserveStock()");
System.out.println(" Step 2: call payment-service.authorizePayment()");
System.out.println(" Step 3: call notification-service.sendConfirmation()");
System.out.println(" If step 2 fails: orchestrator calls inventory-service.releaseStock()");
System.out.println(" All logic in ONE class. Easy to read, debug, and modify.");
}
}
Expected output:
=== Choreography (event-driven) ===
order-service publishes OrderCreated
inventory-service hears OrderCreated, reserves stock, publishes InventoryReserved
payment-service hears InventoryReserved, authorizes payment, publishes PaymentAuthorized
notification-service hears PaymentAuthorized, sends email
Problem: no single place to see the full flow
Problem: if payment fails, who triggers inventory release?
=== Orchestration (central coordinator) ===
OrderSagaOrchestrator controls the entire flow:
Step 1: call inventory-service.reserveStock()
Step 2: call payment-service.authorizePayment()
Step 3: call notification-service.sendConfirmation()
If step 2 fails: orchestrator calls inventory-service.releaseStock()
All logic in ONE class. Easy to read, debug, and modify.
What are the order states in the checkout workflow?
The order moves through a state machine as the Saga progresses. Each forward step transitions the order to the next state. Each compensating step transitions it backward. This state machine makes it easy to understand where any order is in the workflow.
package com.bank.saga;
public class OrderStateMachine {
enum OrderState {
PENDING,
INVENTORY_RESERVED,
PAYMENT_AUTHORIZED,
CONFIRMED,
// Failure states
INVENTORY_FAILED,
PAYMENT_FAILED,
PAYMENT_REFUNDED,
INVENTORY_RELEASED,
CANCELLED
}
record StateTransition(OrderState from, String trigger, OrderState to) {}
public static void main(String[] args) {
var happyPath = java.util.List.of(
new StateTransition(OrderState.PENDING, "reserveInventory() succeeds", OrderState.INVENTORY_RESERVED),
new StateTransition(OrderState.INVENTORY_RESERVED, "authorizePayment() succeeds", OrderState.PAYMENT_AUTHORIZED),
new StateTransition(OrderState.PAYMENT_AUTHORIZED, "sendConfirmation() succeeds", OrderState.CONFIRMED)
);
var failurePath = java.util.List.of(
new StateTransition(OrderState.PENDING, "reserveInventory() fails", OrderState.INVENTORY_FAILED),
new StateTransition(OrderState.INVENTORY_RESERVED, "authorizePayment() fails", OrderState.PAYMENT_FAILED),
new StateTransition(OrderState.PAYMENT_FAILED, "refundPayment()", OrderState.PAYMENT_REFUNDED),
new StateTransition(OrderState.PAYMENT_REFUNDED, "releaseInventory()", OrderState.INVENTORY_RELEASED),
new StateTransition(OrderState.INVENTORY_RELEASED, "markCancelled()", OrderState.CANCELLED)
);
System.out.println("=== Happy Path ===");
happyPath.forEach(t -> System.out.println(" " + t.from() + " -> [" + t.trigger() + "] -> " + t.to()));
System.out.println();
System.out.println("=== Failure + Compensation Path ===");
failurePath.forEach(t -> System.out.println(" " + t.from() + " -> [" + t.trigger() + "] -> " + t.to()));
}
}
Expected output:
=== Happy Path ===
PENDING -> [reserveInventory() succeeds] -> INVENTORY_RESERVED
INVENTORY_RESERVED -> [authorizePayment() succeeds] -> PAYMENT_AUTHORIZED
PAYMENT_AUTHORIZED -> [sendConfirmation() succeeds] -> CONFIRMED
=== Failure + Compensation Path ===
PENDING -> [reserveInventory() fails] -> INVENTORY_FAILED
INVENTORY_RESERVED -> [authorizePayment() fails] -> PAYMENT_FAILED
PAYMENT_FAILED -> [refundPayment()] -> PAYMENT_REFUNDED
PAYMENT_REFUNDED -> [releaseInventory()] -> INVENTORY_RELEASED
INVENTORY_RELEASED -> [markCancelled()] -> CANCELLED
What is the transactional outbox pattern?
The transactional outbox pattern solves the dual-write problem: when a service needs to update its database AND publish an event, doing both atomically is impossible with two different systems (database + Kafka). If the database write succeeds but the Kafka publish fails, the event is lost.
The solution: write the event to an outbox table in the same database transaction as the business data. A separate process (outbox poller or CDC connector) reads the outbox table and publishes events to Kafka. Since the business data and the outbox record are in the same transaction, they either both succeed or both roll back.
package com.bank.saga;
public class TransactionalOutboxDemo {
record OutboxEntry(long id, String aggregateType, String aggregateId,
String eventType, String payload, boolean published) {}
public static void main(String[] args) {
System.out.println("=== Without Outbox (dual-write problem) ===");
System.out.println(" 1. BEGIN TRANSACTION");
System.out.println(" 2. INSERT INTO orders (status='CONFIRMED') -- SUCCESS");
System.out.println(" 3. COMMIT");
System.out.println(" 4. kafkaTemplate.send('order-events', ...) -- FAILS (Kafka down)");
System.out.println(" Result: order is CONFIRMED but event is LOST");
System.out.println();
System.out.println("=== With Outbox Pattern ===");
System.out.println(" 1. BEGIN TRANSACTION");
System.out.println(" 2. INSERT INTO orders (status='CONFIRMED')");
System.out.println(" 3. INSERT INTO outbox (event_type='ORDER_CONFIRMED', payload='{...}')");
System.out.println(" 4. COMMIT (both writes are atomic)");
System.out.println(" 5. Outbox poller reads outbox table every 1 second");
System.out.println(" 6. Poller publishes event to Kafka");
System.out.println(" 7. Poller marks outbox entry as published=true");
System.out.println(" Result: event is guaranteed to be published eventually");
System.out.println();
var entries = java.util.List.of(
new OutboxEntry(1, "Order", "ORD-501", "ORDER_CONFIRMED",
"{\"orderId\":\"ORD-501\",\"amount\":150000}", true),
new OutboxEntry(2, "Order", "ORD-502", "ORDER_CANCELLED",
"{\"orderId\":\"ORD-502\",\"reason\":\"payment failed\"}", false)
);
System.out.println("Outbox table contents:");
for (var entry : entries) {
System.out.println(" id=" + entry.id() + ", type=" + entry.eventType()
+ ", aggregateId=" + entry.aggregateId()
+ ", published=" + entry.published());
}
}
}
Expected output:
=== Without Outbox (dual-write problem) ===
1. BEGIN TRANSACTION
2. INSERT INTO orders (status='CONFIRMED') -- SUCCESS
3. COMMIT
4. kafkaTemplate.send('order-events', ...) -- FAILS (Kafka down)
Result: order is CONFIRMED but event is LOST
=== With Outbox Pattern ===
1. BEGIN TRANSACTION
2. INSERT INTO orders (status='CONFIRMED')
3. INSERT INTO outbox (event_type='ORDER_CONFIRMED', payload='{...}')
4. COMMIT (both writes are atomic)
5. Outbox poller reads outbox table every 1 second
6. Poller publishes event to Kafka
7. Poller marks outbox entry as published=true
Result: event is guaranteed to be published eventually
Outbox table contents:
id=1, type=ORDER_CONFIRMED, aggregateId=ORD-501, published=true
id=2, type=ORDER_CANCELLED, aggregateId=ORD-502, published=false
What is a complete e-commerce checkout project with Saga orchestration?
This project has four Spring Boot services: order-service (Saga orchestrator), inventory-service, payment-service, and notification-service. The order-service coordinates the checkout workflow step by step. If any step fails, the orchestrator runs compensating actions in reverse order.
Project structure
checkout-saga/
├── docker-compose.yml
├── init-db.sql
├── order-service/ (Saga orchestrator)
│ ├── pom.xml
│ └── src/main/java/com/bank/order/
│ ├── OrderServiceApp.java
│ ├── controller/CheckoutController.java
│ ├── service/OrderSagaOrchestrator.java
│ ├── client/InventoryClient.java
│ ├── client/PaymentClient.java
│ ├── client/NotificationClient.java
│ ├── model/Order.java
│ ├── model/OutboxEvent.java
│ ├── dto/CheckoutRequest.java
│ ├── dto/CheckoutResponse.java
│ ├── repository/OrderRepository.java
│ ├── repository/OutboxEventRepository.java
│ ├── config/RestClientConfig.java
│ └── exception/SagaFailedException.java
├── inventory-service/
│ ├── pom.xml
│ └── src/main/java/com/bank/inventory/
│ ├── InventoryServiceApp.java
│ ├── controller/InventoryController.java
│ ├── service/InventoryService.java
│ ├── model/Inventory.java
│ ├── dto/ReserveRequest.java
│ ├── dto/ReleaseRequest.java
│ └── repository/InventoryRepository.java
├── payment-service/
│ ├── pom.xml
│ └── src/main/java/com/bank/payment/
│ ├── PaymentServiceApp.java
│ ├── controller/PaymentController.java
│ ├── service/PaymentService.java
│ ├── model/Payment.java
│ ├── model/IdempotencyRecord.java
│ ├── dto/AuthorizeRequest.java
│ ├── dto/RefundRequest.java
│ └── repository/PaymentRepository.java
└── notification-service/
├── pom.xml
└── src/main/java/com/bank/notification/
├── NotificationServiceApp.java
├── controller/NotificationController.java
└── service/NotificationService.java
docker-compose.yml
version: "3.9"
services:
postgres:
image: postgres:16
environment:
POSTGRES_USER: bank
POSTGRES_PASSWORD: bank123
POSTGRES_DB: bank
ports:
- "5432:5432"
volumes:
- ./init-db.sql:/docker-entrypoint-initdb.d/01-init.sql
order-service:
build: ./order-service
ports:
- "8080:8080"
environment:
SPRING_DATASOURCE_URL: jdbc:postgresql://postgres:5432/bank
INVENTORY_SERVICE_URL: http://inventory-service:8081
PAYMENT_SERVICE_URL: http://payment-service:8082
NOTIFICATION_SERVICE_URL: http://notification-service:8083
depends_on:
- postgres
inventory-service:
build: ./inventory-service
ports:
- "8081:8081"
environment:
SPRING_DATASOURCE_URL: jdbc:postgresql://postgres:5432/bank
depends_on:
- postgres
payment-service:
build: ./payment-service
ports:
- "8082:8082"
environment:
SPRING_DATASOURCE_URL: jdbc:postgresql://postgres:5432/bank
depends_on:
- postgres
notification-service:
build: ./notification-service
ports:
- "8083:8083"
init-db.sql
CREATE TABLE IF NOT EXISTS orders (
id BIGSERIAL PRIMARY KEY,
order_id VARCHAR(50) NOT NULL UNIQUE,
customer_id VARCHAR(20) NOT NULL,
product_id VARCHAR(20) NOT NULL,
quantity INT NOT NULL,
total_amount NUMERIC(15,2) NOT NULL,
status VARCHAR(30) NOT NULL DEFAULT 'PENDING',
idempotency_key VARCHAR(50) NOT NULL UNIQUE,
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
updated_at TIMESTAMP NOT NULL DEFAULT NOW()
);
CREATE TABLE IF NOT EXISTS outbox_events (
id BIGSERIAL PRIMARY KEY,
aggregate_type VARCHAR(50) NOT NULL,
aggregate_id VARCHAR(50) NOT NULL,
event_type VARCHAR(50) NOT NULL,
payload TEXT NOT NULL,
published BOOLEAN NOT NULL DEFAULT FALSE,
created_at TIMESTAMP NOT NULL DEFAULT NOW()
);
CREATE TABLE IF NOT EXISTS inventory (
id BIGSERIAL PRIMARY KEY,
product_id VARCHAR(20) NOT NULL UNIQUE,
product_name VARCHAR(100) NOT NULL,
available_qty INT NOT NULL DEFAULT 0,
reserved_qty INT NOT NULL DEFAULT 0
);
INSERT INTO inventory (product_id, product_name, available_qty) VALUES
('PROD-001', 'Gold Savings Bond', 100),
('PROD-002', 'Fixed Deposit Certificate', 500),
('PROD-003', 'Mutual Fund SIP Unit', 1000);
CREATE TABLE IF NOT EXISTS payments (
id BIGSERIAL PRIMARY KEY,
payment_id VARCHAR(50) NOT NULL UNIQUE,
order_id VARCHAR(50) NOT NULL,
customer_id VARCHAR(20) NOT NULL,
amount NUMERIC(15,2) NOT NULL,
status VARCHAR(20) NOT NULL DEFAULT 'PENDING',
idempotency_key VARCHAR(50) NOT NULL UNIQUE,
created_at TIMESTAMP NOT NULL DEFAULT NOW()
);
CheckoutRequest.java and CheckoutResponse.java
package com.bank.order.dto;
import java.math.BigDecimal;
public record CheckoutRequest(
String customerId,
String productId,
int quantity,
BigDecimal unitPrice,
String idempotencyKey
) {}
package com.bank.order.dto;
import java.math.BigDecimal;
public record CheckoutResponse(
String orderId,
String status,
BigDecimal totalAmount,
String message
) {}
Order.java
package com.bank.order.model;
import jakarta.persistence.Entity;
import jakarta.persistence.Table;
import jakarta.persistence.Id;
import jakarta.persistence.GeneratedValue;
import jakarta.persistence.GenerationType;
import jakarta.persistence.Column;
import java.math.BigDecimal;
import java.time.LocalDateTime;
@Entity
@Table(name = "orders")
public class Order {
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
@Column(name = "order_id", nullable = false, unique = true)
private String orderId;
@Column(name = "customer_id", nullable = false)
private String customerId;
@Column(name = "product_id", nullable = false)
private String productId;
@Column(nullable = false)
private int quantity;
@Column(name = "total_amount", nullable = false)
private BigDecimal totalAmount;
@Column(nullable = false)
private String status;
@Column(name = "idempotency_key", nullable = false, unique = true)
private String idempotencyKey;
@Column(name = "created_at")
private LocalDateTime createdAt = LocalDateTime.now();
@Column(name = "updated_at")
private LocalDateTime updatedAt = LocalDateTime.now();
public Long getId() { return id; }
public String getOrderId() { return orderId; }
public void setOrderId(String orderId) { this.orderId = orderId; }
public String getCustomerId() { return customerId; }
public void setCustomerId(String customerId) { this.customerId = customerId; }
public String getProductId() { return productId; }
public void setProductId(String productId) { this.productId = productId; }
public int getQuantity() { return quantity; }
public void setQuantity(int quantity) { this.quantity = quantity; }
public BigDecimal getTotalAmount() { return totalAmount; }
public void setTotalAmount(BigDecimal totalAmount) { this.totalAmount = totalAmount; }
public String getStatus() { return status; }
public void setStatus(String status) { this.status = status; this.updatedAt = LocalDateTime.now(); }
public String getIdempotencyKey() { return idempotencyKey; }
public void setIdempotencyKey(String key) { this.idempotencyKey = key; }
}
OrderSagaOrchestrator.java (the core of the Saga)
package com.bank.order.service;
import com.bank.order.client.InventoryClient;
import com.bank.order.client.NotificationClient;
import com.bank.order.client.PaymentClient;
import com.bank.order.dto.CheckoutResponse;
import com.bank.order.exception.SagaFailedException;
import com.bank.order.model.Order;
import com.bank.order.repository.OrderRepository;
import org.springframework.stereotype.Service;
import org.springframework.web.client.RestClientException;
import java.math.BigDecimal;
import java.util.UUID;
import java.util.logging.Logger;
@Service
public class OrderSagaOrchestrator {
private static final Logger LOG = Logger.getLogger(OrderSagaOrchestrator.class.getName());
private final OrderRepository orderRepository;
private final InventoryClient inventoryClient;
private final PaymentClient paymentClient;
private final NotificationClient notificationClient;
public OrderSagaOrchestrator(OrderRepository orderRepository,
InventoryClient inventoryClient,
PaymentClient paymentClient,
NotificationClient notificationClient) {
this.orderRepository = orderRepository;
this.inventoryClient = inventoryClient;
this.paymentClient = paymentClient;
this.notificationClient = notificationClient;
}
public CheckoutResponse executeCheckout(String customerId, String productId,
int quantity, BigDecimal unitPrice,
String idempotencyKey) {
var orderId = "ORD-" + UUID.randomUUID().toString().substring(0, 8).toUpperCase();
var totalAmount = unitPrice.multiply(BigDecimal.valueOf(quantity));
// Create order in PENDING state
var order = new Order();
order.setOrderId(orderId);
order.setCustomerId(customerId);
order.setProductId(productId);
order.setQuantity(quantity);
order.setTotalAmount(totalAmount);
order.setStatus("PENDING");
order.setIdempotencyKey(idempotencyKey);
orderRepository.save(order);
LOG.info("Saga started: orderId=" + orderId + ", total=" + totalAmount);
// Step 1: Reserve inventory
try {
inventoryClient.reserveStock(productId, quantity, orderId);
order.setStatus("INVENTORY_RESERVED");
orderRepository.save(order);
LOG.info("Step 1 PASSED: inventory reserved for " + orderId);
} catch (RestClientException ex) {
order.setStatus("INVENTORY_FAILED");
orderRepository.save(order);
LOG.severe("Step 1 FAILED: inventory reservation failed for " + orderId);
return new CheckoutResponse(orderId, "CANCELLED", totalAmount,
"Inventory reservation failed: " + ex.getMessage());
}
// Step 2: Authorize payment
try {
paymentClient.authorizePayment(orderId, customerId, totalAmount, idempotencyKey);
order.setStatus("PAYMENT_AUTHORIZED");
orderRepository.save(order);
LOG.info("Step 2 PASSED: payment authorized for " + orderId);
} catch (RestClientException ex) {
LOG.severe("Step 2 FAILED: payment failed for " + orderId + ". Starting compensation.");
order.setStatus("PAYMENT_FAILED");
orderRepository.save(order);
// Compensate Step 1: release inventory
compensateInventory(order, productId, quantity);
return new CheckoutResponse(orderId, "CANCELLED", totalAmount,
"Payment failed, inventory released: " + ex.getMessage());
}
// Step 3: Send notification
try {
notificationClient.sendConfirmation(orderId, customerId, totalAmount);
order.setStatus("CONFIRMED");
orderRepository.save(order);
LOG.info("Step 3 PASSED: notification sent for " + orderId);
} catch (RestClientException ex) {
// Notification failure is non-critical, order is still confirmed
order.setStatus("CONFIRMED");
orderRepository.save(order);
LOG.warning("Step 3 FAILED (non-critical): notification failed for " + orderId);
}
LOG.info("Saga completed: orderId=" + orderId + ", status=" + order.getStatus());
return new CheckoutResponse(orderId, order.getStatus(), totalAmount, "Checkout completed");
}
private void compensateInventory(Order order, String productId, int quantity) {
try {
inventoryClient.releaseStock(productId, quantity, order.getOrderId());
order.setStatus("CANCELLED");
orderRepository.save(order);
LOG.info("Compensation: inventory released for " + order.getOrderId());
} catch (RestClientException ex) {
order.setStatus("COMPENSATION_FAILED");
orderRepository.save(order);
LOG.severe("COMPENSATION FAILED: inventory release failed for " + order.getOrderId()
+ ". Manual intervention required!");
}
}
}
InventoryClient.java, PaymentClient.java, NotificationClient.java
package com.bank.order.client;
import org.springframework.stereotype.Component;
import org.springframework.web.client.RestClient;
import java.util.Map;
import java.util.logging.Logger;
@Component
public class InventoryClient {
private static final Logger LOG = Logger.getLogger(InventoryClient.class.getName());
private final RestClient restClient;
public InventoryClient(RestClient inventoryRestClient) {
this.restClient = inventoryRestClient;
}
public void reserveStock(String productId, int quantity, String orderId) {
LOG.info("Calling inventory-service: reserve " + quantity + " of " + productId);
restClient.post()
.uri("/inventory/reserve")
.body(Map.of("productId", productId, "quantity", quantity, "orderId", orderId))
.retrieve()
.toBodilessEntity();
}
public void releaseStock(String productId, int quantity, String orderId) {
LOG.info("Calling inventory-service: release " + quantity + " of " + productId);
restClient.post()
.uri("/inventory/release")
.body(Map.of("productId", productId, "quantity", quantity, "orderId", orderId))
.retrieve()
.toBodilessEntity();
}
}
package com.bank.order.client;
import org.springframework.stereotype.Component;
import org.springframework.web.client.RestClient;
import java.math.BigDecimal;
import java.util.Map;
import java.util.logging.Logger;
@Component
public class PaymentClient {
private static final Logger LOG = Logger.getLogger(PaymentClient.class.getName());
private final RestClient restClient;
public PaymentClient(RestClient paymentRestClient) {
this.restClient = paymentRestClient;
}
public void authorizePayment(String orderId, String customerId,
BigDecimal amount, String idempotencyKey) {
LOG.info("Calling payment-service: authorize " + amount + " for " + orderId);
restClient.post()
.uri("/payments/authorize")
.body(Map.of("orderId", orderId, "customerId", customerId,
"amount", amount, "idempotencyKey", idempotencyKey))
.retrieve()
.toBodilessEntity();
}
public void refundPayment(String orderId, String idempotencyKey) {
LOG.info("Calling payment-service: refund for " + orderId);
restClient.post()
.uri("/payments/refund")
.body(Map.of("orderId", orderId, "idempotencyKey", idempotencyKey + "-refund"))
.retrieve()
.toBodilessEntity();
}
}
package com.bank.order.client;
import org.springframework.stereotype.Component;
import org.springframework.web.client.RestClient;
import java.math.BigDecimal;
import java.util.Map;
import java.util.logging.Logger;
@Component
public class NotificationClient {
private static final Logger LOG = Logger.getLogger(NotificationClient.class.getName());
private final RestClient restClient;
public NotificationClient(RestClient notificationRestClient) {
this.restClient = notificationRestClient;
}
public void sendConfirmation(String orderId, String customerId, BigDecimal amount) {
LOG.info("Calling notification-service: confirm " + orderId);
restClient.post()
.uri("/notifications/order-confirmed")
.body(Map.of("orderId", orderId, "customerId", customerId, "amount", amount))
.retrieve()
.toBodilessEntity();
}
}
RestClientConfig.java (order-service)
package com.bank.order.config;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.http.client.SimpleClientHttpRequestFactory;
import org.springframework.web.client.RestClient;
@Configuration
public class RestClientConfig {
private SimpleClientHttpRequestFactory timedFactory() {
var factory = new SimpleClientHttpRequestFactory();
factory.setConnectTimeout(3000);
factory.setReadTimeout(5000);
return factory;
}
@Bean
public RestClient inventoryRestClient(
@Value("\${inventory.service.url:http://localhost:8081}") String url) {
return RestClient.builder().baseUrl(url).requestFactory(timedFactory()).build();
}
@Bean
public RestClient paymentRestClient(
@Value("\${payment.service.url:http://localhost:8082}") String url) {
return RestClient.builder().baseUrl(url).requestFactory(timedFactory()).build();
}
@Bean
public RestClient notificationRestClient(
@Value("\${notification.service.url:http://localhost:8083}") String url) {
return RestClient.builder().baseUrl(url).requestFactory(timedFactory()).build();
}
}
CheckoutController.java
package com.bank.order.controller;
import com.bank.order.dto.CheckoutRequest;
import com.bank.order.dto.CheckoutResponse;
import com.bank.order.service.OrderSagaOrchestrator;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
@RestController
@RequestMapping("/checkout")
public class CheckoutController {
private final OrderSagaOrchestrator orchestrator;
public CheckoutController(OrderSagaOrchestrator orchestrator) {
this.orchestrator = orchestrator;
}
@PostMapping
public ResponseEntity<CheckoutResponse> checkout(@RequestBody CheckoutRequest request) {
var response = orchestrator.executeCheckout(
request.customerId(), request.productId(), request.quantity(),
request.unitPrice(), request.idempotencyKey());
return ResponseEntity.ok(response);
}
}
InventoryService.java
package com.bank.inventory.service;
import com.bank.inventory.model.Inventory;
import com.bank.inventory.repository.InventoryRepository;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import java.util.logging.Logger;
@Service
public class InventoryService {
private static final Logger LOG = Logger.getLogger(InventoryService.class.getName());
private final InventoryRepository repository;
public InventoryService(InventoryRepository repository) {
this.repository = repository;
}
@Transactional
public void reserveStock(String productId, int quantity, String orderId) {
var inventory = repository.findByProductId(productId)
.orElseThrow(() -> new RuntimeException("Product not found: " + productId));
if (inventory.getAvailableQty() < quantity) {
throw new RuntimeException("Insufficient stock for " + productId
+ ": available=" + inventory.getAvailableQty() + ", requested=" + quantity);
}
inventory.setAvailableQty(inventory.getAvailableQty() - quantity);
inventory.setReservedQty(inventory.getReservedQty() + quantity);
repository.save(inventory);
LOG.info("Stock reserved: product=" + productId + ", qty=" + quantity + ", orderId=" + orderId);
}
@Transactional
public void releaseStock(String productId, int quantity, String orderId) {
var inventory = repository.findByProductId(productId)
.orElseThrow(() -> new RuntimeException("Product not found: " + productId));
inventory.setAvailableQty(inventory.getAvailableQty() + quantity);
inventory.setReservedQty(inventory.getReservedQty() - quantity);
repository.save(inventory);
LOG.info("Stock released: product=" + productId + ", qty=" + quantity + ", orderId=" + orderId);
}
}
PaymentService.java (with idempotency)
package com.bank.payment.service;
import com.bank.payment.model.Payment;
import com.bank.payment.repository.PaymentRepository;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import java.math.BigDecimal;
import java.util.UUID;
import java.util.logging.Logger;
@Service
public class PaymentService {
private static final Logger LOG = Logger.getLogger(PaymentService.class.getName());
private final PaymentRepository repository;
public PaymentService(PaymentRepository repository) {
this.repository = repository;
}
@Transactional
public void authorizePayment(String orderId, String customerId,
BigDecimal amount, String idempotencyKey) {
// Idempotency check
if (repository.existsByIdempotencyKey(idempotencyKey)) {
LOG.info("Duplicate payment authorization: key=" + idempotencyKey);
return;
}
var payment = new Payment();
payment.setPaymentId("PAY-" + UUID.randomUUID().toString().substring(0, 8).toUpperCase());
payment.setOrderId(orderId);
payment.setCustomerId(customerId);
payment.setAmount(amount);
payment.setStatus("AUTHORIZED");
payment.setIdempotencyKey(idempotencyKey);
repository.save(payment);
LOG.info("Payment authorized: orderId=" + orderId + ", amount=" + amount);
}
@Transactional
public void refundPayment(String orderId, String idempotencyKey) {
if (repository.existsByIdempotencyKey(idempotencyKey)) {
LOG.info("Duplicate refund: key=" + idempotencyKey);
return;
}
var original = repository.findByOrderIdAndStatus(orderId, "AUTHORIZED")
.orElseThrow(() -> new RuntimeException("No authorized payment for " + orderId));
original.setStatus("REFUNDED");
repository.save(original);
LOG.info("Payment refunded: orderId=" + orderId);
}
}
How to run and test
# Step 1: Start all infrastructure
docker-compose up -d
# Step 2: Build all services
for svc in order-service inventory-service payment-service notification-service; do
cd $svc && mvn clean package -DskipTests && cd ..
done
# Step 3: Start all services
java -jar order-service/target/order-service-1.0.0.jar &
java -jar inventory-service/target/inventory-service-1.0.0.jar &
java -jar payment-service/target/payment-service-1.0.0.jar &
java -jar notification-service/target/notification-service-1.0.0.jar &
# Step 4: Successful checkout (Rajesh buys 2 Gold Savings Bonds at 75000 each)
curl -s -X POST http://localhost:8080/checkout \
-H "Content-Type: application/json" \
-d '{"customerId":"SAV2001","productId":"PROD-001","quantity":2,"unitPrice":75000,"idempotencyKey":"CHK-001"}'
# Step 5: Failed checkout (insufficient stock, product has only 100 units)
curl -s -X POST http://localhost:8080/checkout \
-H "Content-Type: application/json" \
-d '{"customerId":"SAV2002","productId":"PROD-001","quantity":200,"unitPrice":75000,"idempotencyKey":"CHK-002"}'
Expected output (successful checkout):
{
"orderId": "ORD-A1B2C3D4",
"status": "CONFIRMED",
"totalAmount": 150000.00,
"message": "Checkout completed"
}
Logs (order-service):
Saga started: orderId=ORD-A1B2C3D4, total=150000.00
Step 1 PASSED: inventory reserved for ORD-A1B2C3D4
Step 2 PASSED: payment authorized for ORD-A1B2C3D4
Step 3 PASSED: notification sent for ORD-A1B2C3D4
Saga completed: orderId=ORD-A1B2C3D4, status=CONFIRMED
Expected output (failed checkout, insufficient stock):
{
"orderId": "ORD-E5F6G7H8",
"status": "CANCELLED",
"totalAmount": 15000000.00,
"message": "Inventory reservation failed: Insufficient stock..."
}
# Step 6: Simulate payment failure (stop payment-service and try checkout)
kill $(lsof -ti:8082) # Stop payment-service
curl -s -X POST http://localhost:8080/checkout \
-H "Content-Type: application/json" \
-d '{"customerId":"SAV2001","productId":"PROD-002","quantity":1,"unitPrice":50000,"idempotencyKey":"CHK-003"}'
Expected output (payment-service is down, compensation runs):
{
"orderId": "ORD-I9J0K1L2",
"status": "CANCELLED",
"totalAmount": 50000.00,
"message": "Payment failed, inventory released: I/O error..."
}
Logs (order-service):
Saga started: orderId=ORD-I9J0K1L2, total=50000.00
Step 1 PASSED: inventory reserved for ORD-I9J0K1L2
Step 2 FAILED: payment failed for ORD-I9J0K1L2. Starting compensation.
Compensation: inventory released for ORD-I9J0K1L2
What are common mistakes in Saga implementations?
Wrong: no compensating actions (money lost on failure)
package com.bank.saga.wrong;
import java.math.BigDecimal;
public class WrongNoCompensation {
static int inventoryReserved = 0;
static BigDecimal paymentAuthorized = BigDecimal.ZERO;
public static void main(String[] args) {
System.out.println("WRONG: no compensating actions");
System.out.println();
// Step 1: Reserve inventory
inventoryReserved = 2;
System.out.println("Step 1: Reserved 2 units. inventoryReserved=" + inventoryReserved);
// Step 2: Authorize payment
paymentAuthorized = new BigDecimal("150000");
System.out.println("Step 2: Payment authorized. amount=" + paymentAuthorized);
// Step 3: Notification FAILS
System.out.println("Step 3: Notification FAILED (email server down)");
System.out.println();
// No compensation!
System.out.println("Result: inventory is reserved but customer was not notified.");
System.out.println(" If the order is cancelled, inventory stays locked forever.");
System.out.println(" If payment was authorized, money is held indefinitely.");
System.out.println(" NO rollback happened. System is in inconsistent state.");
}
}
Expected output:
WRONG: no compensating actions
Step 1: Reserved 2 units. inventoryReserved=2
Step 2: Payment authorized. amount=150000
Step 3: Notification FAILED (email server down)
Result: inventory is reserved but customer was not notified.
If the order is cancelled, inventory stays locked forever.
If payment was authorized, money is held indefinitely.
NO rollback happened. System is in inconsistent state.
Correct: Saga orchestrator with compensating actions
package com.bank.saga.correct;
import java.math.BigDecimal;
public class CorrectWithCompensation {
static int inventoryReserved = 0;
static BigDecimal paymentAuthorized = BigDecimal.ZERO;
public static void main(String[] args) {
System.out.println("CORRECT: Saga with compensating actions");
System.out.println();
// Step 1: Reserve inventory
inventoryReserved = 2;
System.out.println("Step 1: Reserved 2 units. inventoryReserved=" + inventoryReserved);
// Step 2: Authorize payment FAILS
System.out.println("Step 2: Payment FAILED (payment gateway timeout)");
System.out.println();
// Compensate Step 1: release inventory
System.out.println("Compensating Step 1: releasing 2 units...");
inventoryReserved = 0;
System.out.println(" inventoryReserved=" + inventoryReserved);
System.out.println();
System.out.println("Result: inventory released, order cancelled.");
System.out.println(" System is back to consistent state.");
System.out.println(" Customer can try again.");
}
}
Expected output:
CORRECT: Saga with compensating actions
Step 1: Reserved 2 units. inventoryReserved=2
Step 2: Payment FAILED (payment gateway timeout)
Compensating Step 1: releasing 2 units...
inventoryReserved=0
Result: inventory released, order cancelled.
System is back to consistent state.
Customer can try again.
What should a senior or principal engineer know?
- At the senior level:
- Saga execution log: persist every step (action + result) in a saga_execution_log table. This lets you resume a Saga after a crash. On restart, the orchestrator reads the log and continues from the last successful step.
- Timeout per step: each step should have its own timeout. If inventory-service takes more than 5 seconds, fail the step and start compensation. Do not wait indefinitely.
- Non-critical steps: notification is a non-critical step. If it fails, the order should still be confirmed. Design your Saga to distinguish between critical and non-critical steps.
- At the principal level:
- Saga concurrency: two Sagas might try to reserve the same inventory simultaneously. Use optimistic locking (version column) or pessimistic locking (SELECT FOR UPDATE) in inventory-service to prevent overselling.
- Compensation idempotency: compensating actions must be idempotent too. If releaseStock() is called twice for the same order, it should only release once.
- Dead Saga detection: if the orchestrator crashes mid-Saga, the order is stuck in an intermediate state. Run a periodic job that detects orders in non-terminal states older than 5 minutes and either resumes or cancels them.
- Distributed tracing: use Micrometer Tracing with Zipkin or Jaeger to trace the entire Saga across all four services.
package com.bank.saga;
public class SeniorSagaKnowledge {
public static void main(String[] args) {
System.out.println("Senior/Principal Saga checklist:");
System.out.println(" 1. Persist saga execution log for crash recovery");
System.out.println(" 2. Set timeouts on each saga step");
System.out.println(" 3. Distinguish critical vs non-critical steps");
System.out.println(" 4. Make compensating actions idempotent");
System.out.println(" 5. Use idempotency keys on payment operations");
System.out.println(" 6. Handle concurrent sagas with optimistic locking");
System.out.println(" 7. Dead saga detection job (find stuck orders)");
System.out.println(" 8. Outbox pattern for reliable event publishing");
System.out.println(" 9. Distributed tracing across all saga participants");
System.out.println(" 10. Alert on COMPENSATION_FAILED status (needs manual fix)");
}
}
Expected output:
Senior/Principal Saga checklist:
1. Persist saga execution log for crash recovery
2. Set timeouts on each saga step
3. Distinguish critical vs non-critical steps
4. Make compensating actions idempotent
5. Use idempotency keys on payment operations
6. Handle concurrent sagas with optimistic locking
7. Dead saga detection job (find stuck orders)
8. Outbox pattern for reliable event publishing
9. Distributed tracing across all saga participants
10. Alert on COMPENSATION_FAILED status (needs manual fix)
What are common interview follow-up questions?
- What happens if a compensating action fails? (This is the hardest problem. Log the failure as COMPENSATION_FAILED and alert the operations team. A human must investigate and fix the inconsistency manually. You can also retry the compensation with exponential backoff.)
- Can you mix choreography and orchestration? (Yes. Use orchestration for the main checkout workflow and choreography for downstream events like analytics and audit logging. The orchestrator handles the critical path; choreography handles fire-and-forget events.)
- How do you test a Saga end-to-end? (Use Testcontainers to start all four services and PostgreSQL in Docker. Run the checkout flow with a test client. Verify order state, inventory counts, and payment records. Simulate failures by stopping individual containers mid-Saga.)
- What is the difference between Saga and two-phase commit (2PC)? (2PC locks resources across all participants until the coordinator decides to commit or abort. It provides atomicity but creates tight coupling and is slow. Saga uses eventual consistency with compensating actions. It is faster and more resilient but the system is temporarily inconsistent during execution.)
- How does the Saga pattern work with event sourcing? (Each Saga step produces an event that is appended to the event store. The Saga state is rebuilt by replaying events. This gives you a complete audit trail and enables time-travel debugging.)
Visual Overview
E-Commerce Checkout Saga (Orchestration)
==========================================
┌──────────────────────┐
│ CheckoutController │
│ POST /checkout │
└──────────┬───────────┘
│
▼
┌──────────────────────┐
│ OrderSagaOrchestrator │
│ (coordinates all) │
└──────────┬───────────┘
│
┌───────────────────────┼───────────────────────┐
│ │ │
▼ ▼ ▼
┌─────────────────┐ ┌─────────────────┐ ┌───────────────────┐
│ inventory-service│ │ payment-service │ │notification-service│
│ (port 8081) │ │ (port 8082) │ │ (port 8083) │
│ │ │ │ │ │
│ reserveStock() │ │ authorizePayment()│ │ sendConfirmation() │
│ releaseStock() │ │ refundPayment() │ │ │
└────────┬────────┘ └────────┬─────────┘ └────────────────────┘
│ │
▼ ▼
┌─────────────────┐ ┌─────────────────┐
│ inventory table │ │ payments table │
└─────────────────┘ └─────────────────┘
Happy Path:
PENDING -> INVENTORY_RESERVED -> PAYMENT_AUTHORIZED -> CONFIRMED
Failure at Step 2 (payment fails):
PENDING -> INVENTORY_RESERVED -> PAYMENT_FAILED
Compensation: releaseStock()
PAYMENT_FAILED -> CANCELLED
Failure at Step 1 (no stock):
PENDING -> INVENTORY_FAILED -> CANCELLED (no compensation needed)
Key Takeaway
The Saga pattern breaks a distributed transaction into a sequence of local transactions, each with a compensating action that undoes it if a later step fails. In an e-commerce checkout, the Saga coordinates inventory reservation, payment authorization, and notification delivery across independent microservices. If the payment fails after inventory is reserved, the Saga runs the compensating action to release the inventory and cancel the order.
Orchestration is the preferred approach for complex workflows because the entire flow is visible in one class: OrderSagaOrchestrator. The orchestrator calls each service in sequence, updates the order state after each step, and runs compensating actions in reverse order on failure. Choreography, where each service listens for events and decides what to do next, works for simple flows but becomes hard to trace and debug when there are four or more participants.
In our National Bank of India checkout project, Rajesh bought 2 Gold Savings Bonds at 75,000 rupees each. The orchestrator first called inventory-service to reserve 2 units of PROD-001. Then it called payment-service to authorize 150,000 rupees. Finally, it called notification-service to send a confirmation. Each step updated the order status (PENDING, INVENTORY_RESERVED, PAYMENT_AUTHORIZED, CONFIRMED). When we simulated a payment failure by stopping payment-service, the orchestrator detected the failure, released the inventory, and set the order status to CANCELLED. The system returned to a consistent state.
The transactional outbox pattern complements the Saga by solving the dual-write problem: instead of updating the database and publishing a Kafka event separately (where one can fail), you write both the business data and the outbox record in the same database transaction. A separate poller publishes the event to Kafka later. This guarantees that every state change is eventually published.
At the senior level, make every compensating action idempotent, persist the Saga execution log for crash recovery, and distinguish critical steps (inventory, payment) from non-critical steps (notification). At the principal level, handle concurrent Sagas with optimistic locking, run dead-Saga detection jobs, and use distributed tracing to follow the checkout flow across all four services.
Remember one sentence: a Saga orchestrator coordinates a sequence of local transactions across microservices, updates order state after each step, and runs compensating actions in reverse order when any step fails.
---