Saga pattern: local transactions, compensations, choreography vs orchestration, and long-running consistency

• Saga – a sequence of local transactions across multiple microservices, where each step commits in its own database and triggers the next.
• Compensating transaction – a new forward transaction that semantically undoes a previously committed step, not a database rollback.
• Choreography – each service listens for events and reacts, with no central coordinator.
• Orchestration – a central coordinator sends commands to each service and decides the next step based on replies.
• Eventual consistency – the saga guarantees that all steps eventually succeed or all are compensated, but there is a window of partial completion.

What is the Saga pattern?

The Saga pattern manages data consistency across multiple microservices without a distributed transaction. A saga is a sequence of local transactions, where each step updates one service's database, commits, and then triggers the next step. If any step fails, the saga runs compensating transactions that semantically undo the previously completed steps.

Sagas provide eventual consistency rather than the strict atomicity of a single ACID transaction. There is no shared lock across services and no global commit.

• Each step is a normal ACID transaction within one service.
• After committing, the step publishes an event or sends a command to trigger the next step.
• If a later step fails, compensations run in reverse order.

Why does the Saga pattern exist?

In a monolithic application, a single database transaction can wrap multiple operations atomically. In a microservices architecture, each service has its own database, and a distributed transaction using two-phase commit (2PC) holds locks across all participants for the entire duration, which does not scale and blocks on coordinator failure.

The Saga pattern trades strict atomicity for availability and scalability. No cross-service locks are held, each step commits immediately, and consistency is achieved eventually through compensations. For banking systems at National Bank of India, where transfers span Payments, Loans, and Cards services, sagas keep the system responsive and recoverable.

package com.bank.saga;

import java.math.BigDecimal;

public class PremiumAccountSaga {
    private final PaymentService paymentService;
    private final LoanService loanService;
    private final CardService cardService;

    public PremiumAccountSaga(PaymentService paymentService,
                              LoanService loanService,
                              CardService cardService) {
        this.paymentService = paymentService;
        this.loanService = loanService;
        this.cardService = cardService;
    }

    public void execute(String customerId, BigDecimal joiningFee) {
        String sagaId = java.util.UUID.randomUUID().toString();
        try {
            paymentService.debit(sagaId, customerId, joiningFee);
            loanService.reserveLoan(sagaId, customerId);
            cardService.issueCard(sagaId, customerId);
            System.out.println("Saga " + sagaId + " completed successfully");
        } catch (CardServiceException e) {
            System.out.println("Card step failed, compensating...");
            loanService.cancelReservation(sagaId, customerId);
            paymentService.credit(sagaId, customerId, joiningFee);
            System.out.println("Saga " + sagaId + " compensated");
        }
    }
}

Saga abc-123 step 1: Debited 1200000 from RAJESH-001
Saga abc-123 step 2: Loan reserved for RAJESH-001
Card step failed, compensating...
Saga abc-123 compensate step 2: Loan reservation cancelled
Saga abc-123 compensate step 1: Credited 1200000 to RAJESH-001
Saga abc-123 compensated

How does choreography work?

In choreography, there is no central coordinator. Each service listens for events and reacts by performing its local transaction and publishing its own event for the next service.

Payments ──publishes──→ PaymentDebitedEvent
                              │
                   Loans listens, reserves loan
                              │
Loans ──publishes──→ LoanReservedEvent
                              │
                   Cards listens, issues card

Choreography is simple for short sagas with two or three steps. It avoids a single point of control. But as the number of steps grows, the logic becomes scattered across many services, and there is no single place that shows the whole flow. Debugging a failed saga requires tracing events across multiple logs.

How does orchestration work?

In orchestration, a central coordinator (the saga orchestrator) sends commands to each service and decides the next step based on replies.

package com.bank.saga;

import java.math.BigDecimal;
import java.util.Deque;
import java.util.ArrayDeque;

public class SagaOrchestrator {
    private final Deque<Runnable> compensations = new ArrayDeque<>();

    public void executePremiumAccount(String customerId,
                                      BigDecimal joiningFee) {
        String sagaId = java.util.UUID.randomUUID().toString();
        try {
            // Step 1: debit
            paymentService.debit(sagaId, customerId, joiningFee);
            compensations.push(() ->
                    paymentService.credit(sagaId, customerId, joiningFee));

            // Step 2: reserve loan
            loanService.reserveLoan(sagaId, customerId);
            compensations.push(() ->
                    loanService.cancelReservation(sagaId, customerId));

            // Step 3: issue card
            cardService.issueCard(sagaId, customerId);

        } catch (Exception e) {
            System.out.println("Step failed: " + e.getMessage());
            while (!compensations.isEmpty()) {
                compensations.pop().run();
            }
        }
    }
}

Orchestration is easier to understand, monitor, and change because the whole flow lives in one place. The trade-off is that the orchestrator can become a bottleneck or a single point of failure if not designed carefully.

How is choreography different from orchestration?

| Aspect | Choreography | Orchestration |
|---|---|---|
| Coordinator | None, services react to events | Central orchestrator sends commands |
| Visibility | Flow scattered across services | Flow visible in one place |
| Coupling | Loose, services only know events | Orchestrator knows all services |
| Debugging | Hard, trace events across logs | Easy, inspect orchestrator state |
| Bottleneck risk | None | Orchestrator may become one |
| Best for | Short, simple sagas (2-3 steps) | Complex sagas with many steps |

What are idempotent compensations and the double-compensation problem?

Because messages can be delivered more than once in distributed systems, a compensation might be received twice. If crediting back twelve lakh runs twice, Rajesh gets twenty-four lakh back, which is a serious bug.

To prevent this, make compensations idempotent by recording which saga step and which compensation have already been applied, keyed by a unique saga ID and step ID. A repeated compensation is detected and ignored.

package com.bank.service;

import java.math.BigDecimal;
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;

public class PaymentService {
    private final Set<String> appliedCompensations =
            ConcurrentHashMap.newKeySet();

    public void credit(String sagaId, String customerId,
                       BigDecimal amount) {
        String key = sagaId + ":credit:" + customerId;
        if (!appliedCompensations.add(key)) {
            System.out.println("Duplicate compensation ignored: " + key);
            return;
        }
        // perform the actual credit
        System.out.println("Credited " + amount + " to " + customerId);
    }
}

How is the Saga pattern different from two-phase commit (2PC)?

| Aspect | Saga | Two-Phase Commit |
|---|---|---|
| Locking | No cross-service locks | Holds locks on all participants |
| Consistency | Eventual | Strict atomicity |
| Availability | High, each step commits immediately | Blocks if coordinator fails |
| Scalability | Scales well for microservices | Does not scale, lock contention |
| Undo mechanism | Compensating transactions | Coordinator tells all to abort |
| Suitable for | Loosely coupled microservices | Tightly coupled, same-database systems |

What about non-reversible side effects?

Some actions cannot be undone: sending a real SMS, dispatching a physical card, or making an external payment that is final. Strategies for handling these:

• Order the irreversible step last so it only runs after everything else has succeeded.
• Use a pending or reserved state that is only confirmed at the end, so the irreversible action (printing the card, sending the notification) happens only on final confirmation.
• Issue a counter-action such as a refund rather than a true undo.

// Reserved-then-confirmed approach
cardService.reserveCard(sagaId, customerId);   // no physical card yet
// ... other steps succeed ...
cardService.confirmCard(sagaId, customerId);   // now print and dispatch

What about stuck sagas?

A stuck saga happens when a step neither completes nor fails cleanly, for example because a service is permanently down. You handle this with:

• Timeouts on each step so a missing response triggers compensation after a deadline.
• Retries with exponential backoff for transient failures.
• Dead-letter path for steps that fail after all retries, routed to a human-intervention queue.
• Monitoring and alerting when a saga has been in progress longer than its expected window.

In banking, a silently stuck saga is as dangerous as a failed one, because Rajesh's money can be debited without the corresponding service completing.

What should a senior engineer know?

• Orchestrator scaling. Make the orchestrator stateless by persisting saga state in a database, so any instance can resume a saga. Run multiple orchestrator instances behind a load balancer.
• Saga state persistence. Persist the saga's progress (which steps completed, which compensations are pending) so it survives restarts. Frameworks like Axon, Eventuate, and Spring State Machine provide this.
• Semantic vs syntactic undo. A compensation is not a database rollback. It is a new business operation that achieves the reverse effect. Crediting back is not the same as rolling back a debit.
• Isolation concerns. During a saga, intermediate states are visible to other transactions, which can cause dirty reads. Strategies include semantic locks (marking data as "in-saga") and compensatable reads.

What are common interview follow-up questions?

• How do you guarantee exactly-once compensation?
• What happens if the orchestrator itself crashes mid-saga?
• How do you handle a compensation that itself fails?
• When would you choose choreography over orchestration?
• How do you test a saga end-to-end?