What is the Outbox pattern, why does writing to the DB and publishing to Kafka in one method risk inconsistency, and how do the outbox table + CDC (Debezium) guarantee delivery?

• Dual-write problem – writing to a database and publishing to Kafka in one method touches two independent systems with no shared transaction, so one can succeed while the other fails.
• Outbox pattern – the service writes the business row AND an outbox row inside the SAME local ACID transaction, guaranteeing they commit or roll back together.
• Relay – a separate process reads committed outbox rows and publishes them to Kafka; CDC (Debezium) tails the WAL with zero query load.
• At-least-once – the relay may re-send on crash, so consumers must be idempotent (deduplicate by event ID).
• Ordering – using the aggregate ID as the Kafka key routes all events for one account to the same partition.

What is the dual-write problem?

A naive method does accountRepo.save(account) then kafkaTemplate.send(event). These touch two independent systems with no shared transaction. Four failure modes exist:

1. DB commit succeeds, process crashes before Kafka send: money moved but no event published.
2. Kafka send succeeds, DB transaction rolls back: event published for a transfer that never happened.
3. Kafka send times out but actually succeeded: retry publishes a duplicate.
4. Kafka broker is down: the whole method fails even though the business change was valid.

// WRONG: dual write – DB + Kafka in one method
@Transactional
public void transfer(String fromId, String toId, BigDecimal amount) {
    Account from = accountRepo.findById(fromId).orElseThrow();
    from.debit(amount);
    accountRepo.save(from);  // DB write

    kafkaTemplate.send("transfer.events", new TransferEvent(fromId, toId, amount)); // Kafka write
    // If crash happens HERE → DB committed but event lost!
}

How does the Outbox pattern fix it?

Instead of writing to Kafka, write an outbox row in the SAME database transaction.

package com.bank.service;

import java.math.BigDecimal;
import java.util.UUID;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

@Service
public class TransferService {

    private final AccountRepository accountRepo;
    private final OutboxRepository outboxRepo;

    public TransferService(AccountRepository accountRepo, OutboxRepository outboxRepo) {
        this.accountRepo = accountRepo;
        this.outboxRepo = outboxRepo;
    }

    @Transactional
    public void transfer(String fromId, String toId, BigDecimal amount) {
        Account from = accountRepo.findById(fromId).orElseThrow();
        from.debit(amount);
        accountRepo.save(from);

        // Outbox row in the SAME transaction
        outboxRepo.save(new OutboxEvent(
            UUID.randomUUID().toString(),
            "Transfer",
            fromId, // aggregate ID
            String.format("{\"from\":\"%s\",\"to\":\"%s\",\"amount\":%s}", fromId, toId, amount)
        ));
        // Both rows commit or roll back together – no window of inconsistency
    }
}

How does the relay publish events?

There are two approaches.

Polling publisher

A scheduled job selects unpublished rows, sends them to Kafka, and marks them published.

CDC with Debezium (production-grade)

Debezium tails the database write-ahead log. It sees committed INSERTs into the outbox table and streams them to Kafka. CDC adds zero query load, captures every commit in order, and does not miss rows between poll intervals.

  Debezium CDC Flow
  ┌──────────┐  INSERT outbox row  ┌──────────┐
  │ Service  │ ──────────────────► │PostgreSQL│
  └──────────┘                     │ WAL      │
                                   └────┬─────┘
                                        │ tail WAL
                                   ┌────▼─────┐
                                   │ Debezium │
                                   │ CDC      │
                                   └────┬─────┘
                                        │ publish
                                   ┌────▼─────┐
                                   │  Kafka   │
                                   └──────────┘

Why is the guarantee at-least-once?

The relay may publish an event and crash before marking the row published. On restart it re-publishes the same event. Consumers must be idempotent.

package com.bank.consumer;

public class TransferEventConsumer {

    private final ProcessedEventRepository processedRepo;

    public void handle(TransferEvent event) {
        if (processedRepo.existsById(event.getEventId())) {
            // Duplicate – skip
            return;
        }
        // Process the event
        processedRepo.save(event.getEventId());
    }
}

How do you maintain ordering?

Use the aggregate ID as the Kafka key. All events for one account land on the same partition, preserving per-account order.

How do you prevent the outbox table from growing?

A retention job deletes published rows older than a window (e.g., 7 days). For high volume, partition the outbox table by day and drop old partitions.

What are common mistakes?

// WRONG: dual write to DB and Kafka
accountRepo.save(account);
kafkaTemplate.send(topic, event); // two systems, no shared transaction

// CORRECT: single write to DB with outbox row
accountRepo.save(account);
outboxRepo.save(outboxEvent); // same DB, same transaction

What should a senior engineer know?

• The Debezium outbox event router transform maps outbox columns to the correct Kafka topic and key automatically.
• At-least-once means consumers MUST deduplicate; the dedup check and the business write should be in one transaction.
• For very high volume, partition the outbox table and use parallel relay workers.
• The outbox pattern converts an impossible distributed atomic write into a reliable local transaction plus at-least-once delivery with idempotent consumers.