How to implement rollback in a Saga using orchestration | Microservices

Definition

Saga rollback is the orchestrator running idempotent compensating transactions in reverse for every step that already committed, because there is no shared database rollback across services.

Compensating transaction – A new forward action that semantically undoes a previously committed step: a refund undoes a charge, a release undoes a reservation.
Orchestrator – The central coordinator that defines the step order, tracks Saga state, and drives both forward execution and reverse compensation.
Reverse order – Compensations run in the opposite sequence of the forward steps because later steps may depend on earlier ones.
Idempotency – Each compensation must produce the same result whether executed once or many times, enforced via an idempotency key.
Durable state – The orchestrator persists which steps completed and which compensations are owed, surviving crashes without leaving a half-unwound flow.

Think of National Bank of India processing Rajesh's order with a case manager (the orchestrator) holding a checklist. The manager ticks off each completed step: order created, stock reserved, money charged. If a later step fails, the manager walks back down the checklist in reverse, doing the opposite action for each ticked item: refund the money, release the stock, cancel the order. Because a courier might get lost, the manager makes sure that doing refund twice is harmless, so repeating a correction never causes a double refund.

Understanding the Concepts

What is Saga rollback and why is it not a database ROLLBACK?

Each forward step in a Saga already committed its own local database transaction in its own service. Those commits are permanent and cannot be undone with a SQL ROLLBACK. To reverse a committed step, you run a compensating transaction that performs the inverse business action.

JAVA

package com.bank.saga;

// WRONG: Trying to use database rollback across services
// connection.rollback(); // Only rolls back THIS service's uncommitted work
// It CANNOT undo a committed payment in the payment-service

// CORRECT: Issue a compensating transaction
public interface PaymentService {
    PaymentResult charge(String accountId, java.math.BigDecimal amount, String sagaId);
    RefundResult refund(String accountId, java.math.BigDecimal amount, String sagaId);
}
// refund() is the compensating transaction for charge()

TEXT

compensation is a NEW forward action that semantically negates a previous one:
  charge()  -> compensate with refund()
  reserve() -> compensate with release()
  create()  -> compensate with cancel()

How does the orchestrator define forward steps and compensations?

The orchestrator maintains an ordered list of steps. Each step has a forward action and a corresponding compensation. As the Saga progresses, the orchestrator records which compensations it now owes.

JAVA

package com.bank.saga;

import java.math.BigDecimal;
import java.util.List;

public class TransferSagaOrchestrator {

    private final OrderService orderService;
    private final InventoryService inventoryService;
    private final PaymentService paymentService;
    private final SagaStateRepository sagaRepo;

    public TransferSagaOrchestrator(
            OrderService orderService,
            InventoryService inventoryService,
            PaymentService paymentService,
            SagaStateRepository sagaRepo) {
        this.orderService = orderService;
        this.inventoryService = inventoryService;
        this.paymentService = paymentService;
        this.sagaRepo = sagaRepo;
    }

    public void executeSaga(String sagaId, String accountId, BigDecimal amount) {
        SagaState state = new SagaState(sagaId);

        try {
            // T1: Create order
            orderService.createOrder(sagaId, accountId, amount);
            state.markCompleted("CREATE_ORDER");
            sagaRepo.save(state);

            // T2: Reserve stock
            inventoryService.reserveStock(sagaId, "FD-PRODUCT-001", 1);
            state.markCompleted("RESERVE_STOCK");
            sagaRepo.save(state);

            // T3: Charge payment
            paymentService.charge(accountId, amount, sagaId);
            state.markCompleted("CHARGE_PAYMENT");
            state.setStatus(SagaStatus.COMPLETED);
            sagaRepo.save(state);

            System.out.println("Saga " + sagaId + " completed successfully");

        } catch (Exception e) {
            System.out.println("Step failed: " + e.getMessage() + ". Starting compensation.");
            compensate(state);
        }
    }

    private void compensate(SagaState state) {
        // Compensations run in REVERSE order
        List<String> completedSteps = state.getCompletedSteps();

        for (int i = completedSteps.size() - 1; i >= 0; i--) {
            String step = completedSteps.get(i);
            switch (step) {
                case "CHARGE_PAYMENT" -> paymentService.refund(
                        state.getAccountId(), state.getAmount(), state.getSagaId());
                case "RESERVE_STOCK" -> inventoryService.releaseStock(
                        state.getSagaId(), "FD-PRODUCT-001", 1);
                case "CREATE_ORDER" -> orderService.cancelOrder(state.getSagaId());
            }
            System.out.println("Compensated: " + step);
        }
        state.setStatus(SagaStatus.COMPENSATED);
        sagaRepo.save(state);
    }
}

TEXT

Expected output (when T3 fails):
Step failed: Payment gateway timeout. Starting compensation.
Compensated: RESERVE_STOCK
Compensated: CREATE_ORDER

Why must compensations run in reverse order?

Later steps often depend on earlier ones. If T2 (reserve stock) depends on T1 (order created), you must undo T2 before T1 so you do not try to release stock for a non-existent order. Reverse order mirrors the dependency chain.

TEXT

Forward:  T1 (create order) -> T2 (reserve stock) -> T3 (charge payment)
Reverse:  C2 (release stock) -> C1 (cancel order)
// C3 not needed because T3 failed and never committed

Why must compensations be idempotent?

The orchestrator may retry a compensation after a transient failure, such as a network timeout. Executing a refund twice must have the same effect as doing it once. An idempotency key per compensation prevents double refunds.

JAVA

package com.bank.service;

import java.math.BigDecimal;
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;

public class PaymentServiceImpl implements PaymentService {

    private final Set<String> processedRefunds = ConcurrentHashMap.newKeySet();

    @Override
    public RefundResult refund(String accountId, BigDecimal amount, String sagaId) {
        String idempotencyKey = "REFUND-" + sagaId;

        // Idempotent: if already refunded, return success without acting
        if (!processedRefunds.add(idempotencyKey)) {
            System.out.println("Refund already processed for saga: " + sagaId);
            return new RefundResult(sagaId, "ALREADY_REFUNDED");
        }

        // Process the actual refund
        System.out.println("Refunding " + amount + " to account " + accountId);
        return new RefundResult(sagaId, "REFUNDED");
    }
}

TEXT

Expected output (first call):
Refunding 50000.00 to account SAV2001

Expected output (retry):
Refund already processed for saga: SAGA-001

How does durable state survive orchestrator crashes?

The orchestrator persists the Saga's progress in a database. After a crash and restart, it reads the state, sees which compensations are owed but not yet executed, and resumes compensation.

JAVA

package com.bank.saga;

import jakarta.persistence.*;
import java.util.ArrayList;
import java.util.List;

@Entity
@Table(name = "saga_state")
public class SagaState {

    @Id
    private String sagaId;

    @Enumerated(EnumType.STRING)
    private SagaStatus status = SagaStatus.STARTED;

    @ElementCollection
    @CollectionTable(name = "saga_completed_steps")
    private List<String> completedSteps = new ArrayList<>();

    private String accountId;
    private java.math.BigDecimal amount;

    public SagaState() {}

    public SagaState(String sagaId) {
        this.sagaId = sagaId;
    }

    public void markCompleted(String step) {
        completedSteps.add(step);
    }

    // Getters and setters omitted for brevity
    public String getSagaId() { return sagaId; }
    public SagaStatus getStatus() { return status; }
    public void setStatus(SagaStatus status) { this.status = status; }
    public List<String> getCompletedSteps() { return completedSteps; }
    public String getAccountId() { return accountId; }
    public java.math.BigDecimal getAmount() { return amount; }
}

public enum SagaStatus {
    STARTED, COMPLETED, COMPENSATING, COMPENSATED, FAILED
}

TEXT

Saga state after crash and recovery:
  sagaId: SAGA-001
  status: COMPENSATING
  completedSteps: [CREATE_ORDER, RESERVE_STOCK]
  -> Orchestrator resumes: compensate RESERVE_STOCK, then CREATE_ORDER

What are non-compensatable steps and how do you handle them?

Some actions cannot be undone: sending a physical shipment, wiring money through an external rail, or sending an SMS. These are ordered last in the Saga or treated as pivot points after which the Saga can only go forward.

JAVA

package com.bank.saga;

public class OrderSagaStepOrder {
    // Steps ordered by compensatability:
    // 1. Create order         -> cancel order (compensatable)
    // 2. Reserve stock         -> release stock (compensatable)
    // 3. Charge payment        -> refund payment (compensatable)
    // 4. Ship goods            -> NON-COMPENSATABLE (pivot point)
    //
    // If step 4 fails, we compensate steps 1-3.
    // If step 4 succeeds, we CANNOT roll back, only go forward.
    // Therefore step 4 must be the LAST step.

    public static void main(String[] args) {
        System.out.println("Saga step order:");
        System.out.println("  T1: Create Order (compensatable)");
        System.out.println("  T2: Reserve Stock (compensatable)");
        System.out.println("  T3: Charge Payment (compensatable)");
        System.out.println("  T4: Ship Goods (PIVOT - non-compensatable)");
    }
}

TEXT

Expected output:
Saga step order:
  T1: Create Order (compensatable)
  T2: Reserve Stock (compensatable)
  T3: Charge Payment (compensatable)
  T4: Ship Goods (PIVOT - non-compensatable)

When should you use a workflow engine like Temporal?

A workflow engine provides durable Saga state, automatic retries with backoff, timeout handling, and a dashboard for observing Saga progress. Use one instead of hand-rolling the state persistence and retry loops.

JAVA

package com.bank.saga.temporal;

import io.temporal.workflow.Saga;
import io.temporal.workflow.WorkflowInterface;
import io.temporal.workflow.WorkflowMethod;
import java.math.BigDecimal;

@WorkflowInterface
public interface TransferSagaWorkflow {
    @WorkflowMethod
    void executeTransfer(String accountId, BigDecimal amount);
}

// Implementation with Temporal's built-in Saga support
public class TransferSagaWorkflowImpl implements TransferSagaWorkflow {

    @Override
    public void executeTransfer(String accountId, BigDecimal amount) {
        Saga saga = new Saga(new Saga.Options.Builder().build());

        try {
            // T1: Create order + register compensation
            orderActivities.createOrder(accountId, amount);
            saga.addCompensation(orderActivities::cancelOrder, accountId);

            // T2: Reserve stock + register compensation
            inventoryActivities.reserveStock(accountId);
            saga.addCompensation(inventoryActivities::releaseStock, accountId);

            // T3: Charge payment + register compensation
            paymentActivities.charge(accountId, amount);
            saga.addCompensation(paymentActivities::refund, accountId, amount);

        } catch (Exception e) {
            // Temporal runs all registered compensations in reverse
            saga.compensate();
            throw e;
        }
    }
}

TEXT

Temporal provides:
  - Durable state that survives worker crashes
  - Automatic retries with configurable backoff
  - Compensation execution with built-in reverse ordering
  - Dashboard showing each Saga's current step and state

What are common interview follow-up questions?

Interviewers often ask how orchestration compares to choreography for rollback, and how you handle a compensation that itself fails.

| Aspect | Orchestration Rollback | Choreography Rollback |
|---|---|---|
| Who drives compensation | Central orchestrator | Each service listens for failure events |
| Compensation order | Explicit reverse order | Implicit via event chain |
| Visibility | Single place to see all Saga state | Scattered across service logs |
| Complexity | Orchestrator can become a bottleneck | Harder to reason about and debug |

JAVA

package com.bank.saga;

public class CompensationRetry {
    // When a compensation itself fails:
    // 1. The orchestrator retries with exponential backoff
    // 2. If retries exhaust, alert for manual intervention
    // 3. The Saga stays in COMPENSATING state until resolved
    // 4. A human operator can inspect and complete the compensation

    public static void main(String[] args) {
        System.out.println("Compensation failure handling:");
        System.out.println("  Retry 1: wait 1s");
        System.out.println("  Retry 2: wait 2s");
        System.out.println("  Retry 3: wait 4s");
        System.out.println("  Exhausted: alert ops team, Saga stays in COMPENSATING");
    }
}

TEXT

Expected output:
Compensation failure handling:
  Retry 1: wait 1s
  Retry 2: wait 2s
  Retry 3: wait 4s
  Exhausted: alert ops team, Saga stays in COMPENSATING

Visual Overview

DIAGRAM

  Forward Execution:
  ┌──────────┐    ┌──────────┐    ┌──────────┐
  │ T1: Create│───▶│ T2: Reserve───▶│ T3: Charge│
  │   Order   │    │   Stock   │    │  Payment  │
  └──────────┘    └──────────┘    └─────┬────┘
                                        │ FAILS!
                                        ▼
  Compensation (reverse order):
  ┌──────────┐    ┌──────────┐
  │ C2: Release◀──│ C1: Cancel│
  │   Stock   │    │   Order  │
  └──────────┘    └──────────┘
       ▲                ▲
       │                │
  Orchestrator drives compensations in reverse
  State: COMPENSATING -> COMPENSATED

  Orchestrator State Machine:
  ┌─────────┐   success   ┌───────────┐
  │ STARTED │────────────▶│ COMPLETED │
  └────┬────┘             └───────────┘
       │ failure
       ▼
  ┌──────────────┐  all done  ┌─────────────┐
  │ COMPENSATING │───────────▶│ COMPENSATED │
  └──────────────┘            └─────────────┘

Key Takeaway

Rollback in an orchestrated Saga is not a database rollback. Each forward step has already committed its own local transaction in its own service, so those commits are permanent. To undo a committed step, the orchestrator runs a compensating transaction that performs the inverse business action: a refund to undo a charge, a release to undo a reservation, a cancel to undo an order. Compensation is a new forward action that semantically negates a previous one.

The orchestrator defines the ordered forward steps and, for each step that needs undoing, a corresponding compensation. As it executes, it records in durable state which compensations it now owes. When a step fails, it stops going forward and runs the owed compensations in reverse order, because later steps depend on earlier ones. Compensations must be idempotent so the orchestrator can safely retry after transient failures, and the durable state ensures that even after a crash, the orchestrator resumes and completes pending compensations rather than leaving a half-unwound flow.

At National Bank of India, Pankaj implemented this for the transfer Saga. When shipment scheduling failed after Rajesh's payment had been charged, the orchestrator drove the rollback in reverse: it refunded the payment, released the stock, and cancelled the order, each compensation idempotent with a saga-keyed idempotency check. In production the team uses Temporal so the durable state, automatic retries, and compensation visibility come built-in rather than hand-rolled.

Remember one sentence: Saga rollback under orchestration is the orchestrator executing idempotent, retryable compensating transactions in reverse order for every step that already committed, because committed local transactions across services cannot be database-rolled-back.

---