Definition
Kafka transactions and the transactional outbox pattern solve different atomicity problems.
- Kafka transaction – groups producer sends and offset commits into one atomic unit; all commit or all abort.
- transaction-id-prefix – the Spring property that enables transactional producers and zombie fencing.
- Zombie fencing – a restarted producer gets a new epoch; the broker rejects writes from the old stale instance.
- Read-process-write – consuming from one topic, processing, and producing to another with exactly-once by tying offsets to the transaction.
- Transactional outbox – writes the event to an outbox table in the same local DB transaction as the business change; a relay publishes to Kafka.
- Dual write – writing to both DB and Kafka independently; a crash between them leaves the systems disagreeing.
A Kafka transaction at National Bank of India is like a sealed batch of outgoing letters plus a note saying how far you have read the incoming mail: either the whole batch is mailed and the note updated together, or nothing happens. That works when everything is inside the Kafka post office. But the bank's ledger is a separate building (the database), and the post office cannot reach inside the ledger to make both commit together. The transactional outbox fixes this by writing the letter into a special tray inside the ledger building in the same ledger transaction, and a separate courier later carries it to the post office.
Understanding the Concepts
What is a Kafka transaction?
A Kafka transaction groups a set of producer sends and an offset commit into one atomic unit. You start it, send records to any number of topics, optionally send consumed offsets, and commit or abort. If you commit, all records become visible and offsets advance; if you abort, nothing happens.
package com.bank.kafka;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.stereotype.Service;
@Service
public class TransactionalProducerDemo {
private final KafkaTemplate<String, String> template;
public TransactionalProducerDemo(KafkaTemplate<String, String> template) {
this.template = template;
}
public void transferFunds() {
template.executeInTransaction(t -> {
t.send("debit.events", "SAV2001", "{\"amount\":50000}");
t.send("credit.events", "SAV3001", "{\"amount\":50000}");
// Both sends commit together or both abort
return null;
});
System.out.println("Both debit and credit published atomically");
}
}
Both debit and credit published atomically
How do you enable Kafka transactions in Spring?
Set spring.kafka.producer.transaction-id-prefix to any stable prefix. This makes KafkaTemplate transactional and registers a KafkaTransactionManager. Use @Transactional or executeInTransaction.
# application.properties
spring.kafka.producer.transaction-id-prefix=settlement-tx-
spring.kafka.consumer.isolation.level=read_committed
spring.kafka.consumer.properties.enable.auto.commit=false
package com.bank.kafka;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
@Service
public class DeclarativeTransactionDemo {
private final KafkaTemplate<String, String> template;
public DeclarativeTransactionDemo(KafkaTemplate<String, String> template) {
this.template = template;
}
@Transactional // Kafka transaction wraps the whole method
public void processAndPublish() {
template.send("settlements", "SAV2001", "{\"status\":\"settled\"}");
template.send("notifications", "SAV2001", "{\"msg\":\"Transfer complete\"}");
// Both commit on method return, or both abort on exception
}
}
# On success: both records visible to read_committed consumers
# On exception: both records invisible, offsets unchanged
How does read-process-write achieve exactly-once?
The consumer reads a record, processes it, produces output, and the consumed offset is included in the same transaction as the produced output. Either the output is produced and the offset advances, or neither happens.
Read-process-write pipeline:
1. Consumer reads from input-topic (offset 42)
2. Process: transform the record
3. Producer sends to output-topic
4. Consumer offset 42 is sent to the Kafka transaction
5. Transaction COMMIT: output visible + offset advances to 43
or ABORT: output invisible + offset stays at 42 (re-read)
Downstream consumers read output-topic with isolation.level=read_committed
What is zombie fencing?
The transactional ID enables zombie fencing. A restarted producer gets a new epoch, and the broker rejects writes from the old stale instance. This prevents a hung former leader from corrupting the transaction.
package com.bank.kafka;
public class ZombieFencingDemo {
public static void main(String[] args) {
System.out.println("Producer-A starts with txn-id='settlement-tx-0', epoch=1");
System.out.println("Producer-A hangs (network partition)");
System.out.println("Producer-B starts with same txn-id, gets epoch=2");
System.out.println("Producer-A tries to write -> broker rejects (stale epoch)");
System.out.println("Only Producer-B (latest epoch) can write -> no corruption");
}
}
Producer-A starts with txn-id='settlement-tx-0', epoch=1
Producer-A hangs (network partition)
Producer-B starts with same txn-id, gets epoch=2
Producer-A tries to write -> broker rejects (stale epoch)
Only Producer-B (latest epoch) can write -> no corruption
Why can't Kafka transactions span a database?
A Kafka transaction is confined to Kafka. There is no shared two-phase commit coordinator between the Kafka broker and a relational database. A DB write and a Kafka publish wrapped in the same method are NOT atomic.
package com.bank.kafka;
public class DualWriteProblemDemo {
public static void main(String[] args) {
// WRONG: dual write -- not atomic across DB and Kafka
System.out.println("1. DB: debit Rajesh's account");
System.out.println("2. Kafka: send TransferCompleted event");
System.out.println("CRASH between step 1 and step 2!");
System.out.println("Result: DB debited but event never published");
System.out.println("Downstream services never hear about the transfer");
}
}
1. DB: debit Rajesh's account
2. Kafka: send TransferCompleted event
CRASH between step 1 and step 2!
Result: DB debited but event never published
Downstream services never hear about the transfer
How does the transactional outbox pattern solve the dual write?
In the same local DB transaction that does the business change, insert the event into an outbox table. A separate relay (CDC like Debezium or a poller) reads the outbox and publishes to Kafka.
package com.bank.service;
import java.math.BigDecimal;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
@Service
public class TransferService {
@Transactional // local DB transaction only
public void transfer(String fromAccount, String toAccount, BigDecimal amount) {
// Business tables
accountRepository.debit(fromAccount, amount);
accountRepository.credit(toAccount, amount);
// Outbox table -- in the SAME DB transaction
outboxRepository.save(new OutboxEvent(
"TransferCompleted",
fromAccount,
"{\"from\":\"" + fromAccount + "\",\"to\":\"" + toAccount
+ "\",\"amount\":" + amount + "}"
));
// Debit, credit, and outbox row commit TOGETHER atomically
}
}
// Relay (Debezium CDC) reads outbox table and publishes to Kafka
DB transaction: debit SAV2001 + credit SAV3001 + outbox insert -> COMMIT
Relay: reads outbox row -> publishes to Kafka -> marks row sent
When do you use Kafka transactions vs the outbox?
Kafka transactions are right when both the input and output are Kafka topics. The outbox is right whenever a database is one of the two systems.
| Scenario | Solution | |---|---| | Kafka topic -> process -> Kafka topic | Kafka transaction (exactly-once within Kafka) | | DB write + Kafka publish | Transactional outbox (atomicity via local DB transaction) | | DB write + DB write | Local DB transaction (standard ACID) |
# Rule of thumb:
# Both ends Kafka? -> Kafka transaction
# One end is a database? -> Transactional outbox
# Both ends databases? -> Local or distributed DB transaction
What should a senior engineer know?
The outbox relay is at-least-once: a crash after publishing but before marking the row sent causes a republish. Consumers must be idempotent, keyed by event ID. Downstream consumers of Kafka transactions must use isolation.level=read_committed to see only committed output. Zombie fencing protects against stale producers but adds a small overhead per transaction.
package com.bank.kafka;
public class OutboxRelayGuaranteeDemo {
public static void main(String[] args) {
System.out.println("Relay publishes outbox row to Kafka");
System.out.println("CRASH before marking row as sent");
System.out.println("Restart: relay publishes the same row AGAIN");
System.out.println("Consumer must be idempotent (primary key = event_id)");
System.out.println("Trade-off: guaranteed eventual delivery with tiny latency");
}
}
Relay publishes outbox row to Kafka
CRASH before marking row as sent
Restart: relay publishes the same row AGAIN
Consumer must be idempotent (primary key = event_id)
Trade-off: guaranteed eventual delivery with tiny latency
Visual Overview
Kafka Transaction (both ends Kafka):
Input Topic ──→ Consumer ──→ Process ──→ Producer ──→ Output Topic
│ │
└── offset commit ───────┘
(same Kafka transaction)
COMMIT: output visible + offset advances
ABORT: output invisible + offset unchanged
Transactional Outbox (DB + Kafka):
┌────────────────────────────────────────────┐
│ Local DB Transaction │
│ 1. Debit SAV2001 │
│ 2. Credit SAV3001 │
│ 3. INSERT INTO outbox (event) │
│ ─── all commit together atomically ─── │
└────────────────────────────────────────────┘
│
▼
┌────────────────────────────────────────────┐
│ Relay (Debezium CDC or Poller) │
│ Read outbox row ──→ Publish to Kafka │
│ Mark row as sent │
│ (at-least-once: consumer must be idempotent)│
└────────────────────────────────────────────┘
Key Takeaway
Kafka transactions make a set of producer sends and a consumer offset commit atomic within Kafka. In Spring you enable them by setting the producer transaction-id-prefix, which makes KafkaTemplate transactional and registers a Kafka transaction manager. For read-process-write pipelines where both ends are Kafka topics, tying the consumed offsets to the produced output gives exactly-once between topics.
The critical limitation is that Kafka transactions cannot span a relational database. A DB write plus a Kafka publish wrapped in the same method is a dual write, and a crash between them leaves the two systems disagreeing. At National Bank of India this bit Pankaj's team when a crash debited Rajesh's balance but lost the TransferCompleted event.
The transactional outbox pattern solves this: in the same local DB transaction that debits and credits, insert the event into an outbox table so everything commits together atomically. A separate relay (Debezium CDC or a poller) publishes the outbox rows to Kafka. The relay is at-least-once, so consumers must be idempotent, keyed by event ID.
Remember one sentence: Kafka transactions are right when both ends are Kafka, and the transactional outbox is right whenever a database is one of the two systems.
---