Definition
- At-most-once – each message is delivered zero or one time; messages may be lost but are never duplicated, caused by committing the offset before processing or using
acks=0. - At-least-once – each message is delivered one or more times; messages are never lost but may be duplicated, caused by processing first and committing the offset afterward.
- Exactly-once – each message is effectively processed one and only one time, with neither loss nor duplication, achieved through idempotent producers and Kafka transactions.
- Idempotent producer –
enable.idempotence=true, which makes the broker detect and discard duplicate sends from retries using producer id and sequence numbers. - Kafka transactions – the ability to atomically write to multiple partitions and commit consumer offsets, so either everything commits or nothing does.
- Idempotent consumer – an application-level pattern where the consumer checks for duplicates (unique constraint, processed-id check) before applying side effects.
Imagine National Bank of India mailing transaction receipts to Rajesh. At-most-once is like mailing a receipt and never following up: if the post loses it, Rajesh never gets it, but he never gets two. At-least-once is like mailing a receipt and re-sending if unsure it arrived: Rajesh definitely gets it, but he might receive duplicates. Exactly-once is a careful system where Rajesh gets the receipt once and only once, no matter what goes wrong.
Understanding the Concepts
What are the three delivery semantics?
Delivery semantics describe the guarantee a messaging system makes about how many times a message is delivered and processed in the presence of failures. The semantic you get depends on how you configure producer acknowledgments and retries, and when the consumer commits its offset relative to processing.
package com.bank.semantics;
public class DeliverySemanticsOverview {
public static void main(String[] args) {
System.out.println("Delivery Semantics:");
System.out.println();
System.out.println("At-most-once: 0 or 1 delivery (may LOSE)");
System.out.println("At-least-once: 1 or more (may DUPLICATE)");
System.out.println("Exactly-once: exactly 1 (neither)");
System.out.println();
System.out.println("Banking choice by data type:");
System.out.println(" Metrics: at-most-once (loss OK)");
System.out.println(" Transfers: at-least-once + idempotency");
System.out.println(" Kafka-to-Kafka pipelines: exactly-once");
}
}
Expected output:
At-most-once: may lose, never duplicate
At-least-once: may duplicate, never lose
Exactly-once: neither loss nor duplication
What causes at-most-once delivery?
At-most-once happens when you commit or acknowledge before safely processing. On the consumer side, if a consumer commits its offset immediately after receiving a record but before processing it, and then crashes, the record is marked as done and is never reprocessed. On the producer side, acks=0 means the producer fires and forgets without waiting for the broker.
package com.bank.semantics;
import org.apache.kafka.clients.consumer.ConsumerRecord;
// DANGEROUS for banking: at-most-once consumer
public class AtMostOnceConsumer {
public void processRecord(ConsumerRecord<String, String> record) {
// Step 1: Commit offset FIRST (at-most-once)
// consumer.commitSync();
System.out.println("Offset committed: " + record.offset());
// Step 2: Process the record
// If crash happens HERE, record is lost (offset already committed)
System.out.println("Processing transfer: " + record.value());
// Transfer never applied, but Kafka thinks it is done
}
}
At-most-once flow:
1. Receive record (offset 42)
2. Commit offset 42 -> Kafka marks it done
3. CRASH before processing
4. Restart -> resumes from offset 43
5. Record 42 LOST (never processed)
What causes at-least-once delivery?
At-least-once happens when you process first and commit afterward. The consumer processes the record and only then commits the offset. If it crashes after processing but before committing, the record is redelivered after restart and processed again.
package com.bank.semantics;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.kafka.support.Acknowledgment;
import org.springframework.stereotype.Component;
@Component
public class AtLeastOnceConsumer {
// enable.auto.commit=false, ack-mode=MANUAL
@KafkaListener(topics = "transfers", groupId = "settlement-group")
public void processTransfer(ConsumerRecord<String, String> record,
Acknowledgment ack) {
// Step 1: Process the record FIRST
System.out.println("Processing transfer: " + record.key()
+ " offset=" + record.offset());
// Apply debit, update ledger...
// Step 2: Commit offset AFTER processing
ack.acknowledge();
System.out.println("Offset committed: " + record.offset());
// If crash between step 1 and step 2:
// Record redelivered -> processed AGAIN (duplicate)
}
}
At-least-once flow:
1. Receive record (offset 42)
2. Process transfer (debit applied)
3. CRASH before commit
4. Restart -> resumes from offset 42
5. Record 42 REDELIVERED -> processed AGAIN (duplicate debit)
6. Solution: idempotent consumer
How do you make the consumer idempotent?
An idempotent consumer checks whether a record has already been processed before applying the side effect. The two common techniques are a processed-id check in application code and a unique constraint in the database.
package com.bank.service;
import org.springframework.stereotype.Service;
import java.math.BigDecimal;
@Service
public class IdempotentSettlementService {
private final SettlementRepository repo;
public IdempotentSettlementService(SettlementRepository repo) {
this.repo = repo;
}
public void settleTransfer(String transferId, String accountId,
BigDecimal amount) {
// Check 1: Application-level idempotency check
if (repo.existsByTransferId(transferId)) {
System.out.println("Duplicate detected, skipping: " + transferId);
return;
}
// Process the transfer
repo.save(transferId, accountId, amount);
System.out.println("Settled transfer: " + transferId
+ " account=" + accountId + " amount=" + amount);
// Check 2: UNIQUE constraint on transfer_id column
// prevents duplicate even if app check has a race condition
}
}
interface SettlementRepository {
boolean existsByTransferId(String transferId);
void save(String transferId, String accountId, BigDecimal amount);
}
First delivery:
Settled transfer: TRF-001 account=SAV2001 amount=50000.00
Redelivery after crash:
Duplicate detected, skipping: TRF-001
-> Rajesh never double-debited
What causes exactly-once delivery?
Exactly-once requires coordinated features. On the producer side, idempotence (enable.idempotence=true) makes the broker detect and discard duplicate sends. On top of that, Kafka transactions let a producer write to multiple partitions and commit the consumer offsets atomically.
package com.bank.config;
import java.util.Properties;
public class ExactlyOnceProducerConfig {
public static Properties createConfig() {
Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("key.serializer",
"org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer",
"org.apache.kafka.common.serialization.StringSerializer");
// Exactly-once producer settings
props.put("enable.idempotence", "true");
props.put("acks", "all");
props.put("transactional.id", "transfer-enrichment-tx");
System.out.println("Exactly-once producer configured:");
System.out.println(" enable.idempotence=true");
System.out.println(" acks=all");
System.out.println(" transactional.id=transfer-enrichment-tx");
return props;
}
public static void main(String[] args) {
createConfig();
}
}
Expected output:
Exactly-once producer: idempotence + acks=all + transactional.id
Broker deduplicates retries, transactions make writes atomic.
How is at-least-once different from exactly-once? (comparison table)
| Feature | At-Least-Once | Exactly-Once |
|---|---|---|
| Message loss | Never | Never |
| Duplicates | Possible (consumer handles) | Never (Kafka handles) |
| Offset commit | Manual, after processing | Atomic with writes (transaction) |
| Producer config | acks=all, retries | acks=all, idempotence, transactional.id |
| Consumer config | enable.auto.commit=false | isolation.level=read_committed |
| Side effects | Need app-level idempotency | Kafka-to-Kafka: native; external: idempotent |
| Throughput | Higher | Lower (transaction overhead) |
| Complexity | Simple + idempotent consumer | More complex (transactions) |
package com.bank.comparison;
public class AtLeastOnceVsExactlyOnce {
public static void main(String[] args) {
System.out.println("At-Least-Once:");
System.out.println(" Process -> commit -> done");
System.out.println(" Crash between process and commit -> redelivery");
System.out.println(" Consumer must be idempotent");
System.out.println(" Higher throughput, simpler setup");
System.out.println();
System.out.println("Exactly-Once:");
System.out.println(" Begin transaction -> process -> commit offset + writes");
System.out.println(" Crash -> transaction aborted -> no partial writes");
System.out.println(" No app-level dedup needed (for Kafka-to-Kafka)");
System.out.println(" Lower throughput, more complex setup");
}
}
At-least-once: simple + idempotent consumer (banking default)
Exactly-once: transactions (Kafka-to-Kafka pipelines)
What are common mistakes with delivery semantics?
The most common mistake is assuming at-least-once means no duplicates. It means no loss, but duplicates will happen during rebalances and crashes. Another mistake is using exactly-once transactions for external database writes, where they do not help.
package com.bank.mistakes;
// WRONG: Assuming at-least-once prevents duplicates.
public class WrongAssumption {
public static void main(String[] args) {
System.out.println("WRONG: at-least-once = no duplicates");
System.out.println(" Reality: crash after processing, before commit");
System.out.println(" -> record redelivered -> processed AGAIN");
System.out.println(" -> Rajesh debited TWICE");
System.out.println();
System.out.println("CORRECT: at-least-once + idempotent consumer");
System.out.println(" -> redelivery detected by transfer id check");
System.out.println(" -> duplicate skipped -> Rajesh debited ONCE");
}
}
WRONG: at-least-once without idempotency = duplicate debits
CORRECT: at-least-once + idempotent consumer = effectively once
What are the edge cases with exactly-once?
True exactly-once is native only to Kafka-to-Kafka flows. When the side effect is an external system like a database, the Kafka transaction cannot include the database write. You achieve effectively-once by making the consumer idempotent.
package com.bank.edge;
public class ExactlyOnceEdgeCases {
public static void main(String[] args) {
System.out.println("Exactly-Once Edge Cases:");
System.out.println();
System.out.println("Kafka-to-Kafka: true exactly-once");
System.out.println(" Read from topic A -> enrich -> write to topic B");
System.out.println(" Transaction covers offset commit + write to B");
System.out.println();
System.out.println("Kafka-to-Database: effectively-once");
System.out.println(" Read from topic A -> write to PostgreSQL");
System.out.println(" Kafka transaction cannot include DB write");
System.out.println(" Solution: idempotent consumer (unique constraint)");
System.out.println();
System.out.println("Kafka-to-External-API: effectively-once");
System.out.println(" Read from topic A -> call REST API");
System.out.println(" Solution: idempotency key in API request");
}
}
Kafka-to-Kafka: true exactly-once (transactions)
Kafka-to-Database: effectively-once (idempotent consumer)
Kafka-to-External-API: effectively-once (idempotency key)
What should a senior engineer know?
Exactly-once has real costs: transactions add latency and throughput overhead. In most banking systems the pragmatic choice is at-least-once plus an idempotent consumer, which gives effectively-once results for database writes without the transactional overhead. Reserve Kafka transactions for genuine Kafka-to-Kafka pipelines.
package com.bank.senior;
public class DeliverySemanticsSeniorGuide {
public static void main(String[] args) {
System.out.println("Senior Decision Guide:");
System.out.println();
System.out.println("Metrics/telemetry: at-most-once (loss OK)");
System.out.println("Transfers/payments: at-least-once + idempotency");
System.out.println("Kafka enrichment: exactly-once (transactions)");
System.out.println();
System.out.println("Why not exactly-once everywhere?");
System.out.println(" 1. Transaction overhead (latency + throughput)");
System.out.println(" 2. Only covers Kafka-to-Kafka writes");
System.out.println(" 3. External writes still need idempotency");
System.out.println(" 4. Complexity of transactional.id management");
}
}
Expected output:
Metrics: at-most-once
Transfers: at-least-once + idempotency (banking default)
Kafka enrichment: exactly-once (transactions)
Visual Overview
Delivery Semantics: Offset Commit Timing
=========================================
At-Most-Once (commit BEFORE processing):
Receive record -> Commit offset -> Process
│ │
CRASH HERE Record LOST
(offset committed, processing skipped)
At-Least-Once (commit AFTER processing):
Receive record -> Process -> Commit offset
│ │
CRASH HERE Record REDELIVERED
(processed but offset not committed)
Exactly-Once (atomic transaction):
Begin TX -> Process -> Write output + Commit offset -> Commit TX
│
CRASH HERE
TX aborted, nothing committed
Restart: reprocess from old offset
No duplicate output
Key Takeaway
Delivery semantics describe how many times a message is processed when failures happen. At-most-once commits the offset before processing, so a crash loses the record. At-least-once processes first and commits afterward, so a crash causes redelivery and duplication. Exactly-once uses idempotent producers and Kafka transactions to ensure neither loss nor duplication, but only for Kafka-to-Kafka flows.
At National Bank of India, the choice depends on the data. Dashboard metrics use at-most-once because losing a few points is harmless and they refresh constantly. Transfer events use at-least-once with manual offset commits and an idempotent consumer keyed by transfer id, which gives effectively-once results so Rajesh is never double-debited. Kafka-to-Kafka enrichment pipelines use full exactly-once with transactions.
The crucial real-world caveat is that true exactly-once only covers Kafka-internal writes. When the side effect is a database or an external API, the Kafka transaction cannot include that external operation, so you still need application-level idempotency. This is why at-least-once plus an idempotent consumer is the pragmatic banking default.
Remember one sentence: at-least-once with an idempotent consumer is the banking default because it guarantees no loss while preventing duplicate effects through application-level deduplication.
---