Definition
- Idempotency key – Client sends a unique key per operation; server stores the key-to-response mapping and replays it on retry.
- DB unique constraint – A unique index on a natural business key lets the database reject duplicates atomically.
- State-based validation – Check the current state and skip if already in the target state; natural for state machines.
- Distributed lock – Acquire a Redis lock (
SET NX PX ttl) so only one request wins; for non-DB scarce resources. - Preference order – Unique constraint > state check > idempotency key > distributed lock, by cost and simplicity.
At National Bank of India there are four ways staff avoid doing something twice. The idempotency key is giving Rajesh a receipt number so if he asks again they hand back the same receipt. The unique constraint is the ledger refusing a second entry with the same cheque number. State-based validation is the clerk seeing the account is already closed and not closing it again. The distributed lock is a single festival coupon counter where only one teller at a time can hand out the last coupon.
Understanding the Concepts
When do you use an idempotency key?
Use it when a client command may be retried and you must return the same outcome, classically payments. The client generates a unique key and sends it on every attempt.
package com.bank.idempotency;
import org.springframework.data.redis.core.StringRedisTemplate;
import org.springframework.stereotype.Service;
import java.math.BigDecimal;
import java.time.Duration;
@Service
public class PaymentService {
private final StringRedisTemplate redis;
public PaymentService(StringRedisTemplate redis) {
this.redis = redis;
}
public String charge(String idempotencyKey, String accountId, BigDecimal amount) {
// Atomic claim: SET NX ensures only first caller wins
Boolean claimed = redis.opsForValue()
.setIfAbsent("idem:" + idempotencyKey, "IN_PROGRESS", Duration.ofHours(24));
if (Boolean.FALSE.equals(claimed)) {
// Key exists: return stored response
String stored = redis.opsForValue().get("idem:" + idempotencyKey);
System.out.println("Replaying stored result for key: " + idempotencyKey);
return stored;
}
// Process the charge
String result = "{\"txnId\":\"TXN-001\",\"status\":\"CHARGED\"}";
redis.opsForValue().set("idem:" + idempotencyKey, result, Duration.ofHours(24));
System.out.println("Charged " + amount + " to " + accountId);
return result;
}
}
Expected output (first call):
Charged 50000.00 to SAV2001
Expected output (retry):
Replaying stored result for key: IK-20240115-001
When do you use a DB unique constraint?
Use it when the goal is preventing duplicate records and the database can enforce it atomically.
package com.bank.idempotency;
import org.springframework.dao.DuplicateKeyException;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import java.math.BigDecimal;
@Service
public class LedgerService {
private final LedgerEntryRepository ledgerRepo;
public LedgerService(LedgerEntryRepository ledgerRepo) {
this.ledgerRepo = ledgerRepo;
}
@Transactional
public String recordEntry(String transactionRef, String accountId, BigDecimal amount) {
try {
// UNIQUE(transaction_reference) in the database
ledgerRepo.save(new LedgerEntry(transactionRef, accountId, amount));
System.out.println("Ledger entry recorded: " + transactionRef);
return "RECORDED";
} catch (DuplicateKeyException e) {
System.out.println("Duplicate entry rejected: " + transactionRef);
return "ALREADY_RECORDED";
}
}
}
-- Table with unique constraint
CREATE TABLE ledger_entries (
id BIGSERIAL PRIMARY KEY,
transaction_reference VARCHAR(50) NOT NULL,
account_id VARCHAR(20) NOT NULL,
amount DECIMAL(15,2) NOT NULL,
created_at TIMESTAMP DEFAULT NOW(),
CONSTRAINT uk_transaction_ref UNIQUE (transaction_reference)
);
Expected output (first call):
Ledger entry recorded: TXN-REF-001
Expected output (retry):
Duplicate entry rejected: TXN-REF-001
When do you use state-based validation?
Use it when the operation is a state transition and repeating it is a no-op if the resource is already in the target state.
package com.bank.idempotency;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
@Service
public class AccountClosureService {
private final AccountRepository accountRepo;
public AccountClosureService(AccountRepository accountRepo) {
this.accountRepo = accountRepo;
}
@Transactional
public String closeAccount(String accountId) {
Account account = accountRepo.findById(accountId).orElseThrow();
if (account.getStatus() == AccountStatus.CLOSED) {
System.out.println("Account " + accountId + " already closed, skipping");
return "ALREADY_CLOSED";
}
// Conditional update: atomic check-and-act
int updated = accountRepo.closeIfNotClosed(accountId);
// UPDATE accounts SET status='CLOSED' WHERE id=? AND status<>'CLOSED'
if (updated == 0) {
System.out.println("Account " + accountId + " closed by another thread");
return "ALREADY_CLOSED";
}
System.out.println("Account " + accountId + " closed successfully");
return "CLOSED";
}
}
Expected output (first call):
Account SAV2001 closed successfully
Expected output (second call):
Account SAV2001 already closed, skipping
When do you use a distributed lock?
Use it when a single scarce resource must be handed out exactly once under high concurrency across many instances, and the resource is not a DB row you can constrain.
package com.bank.idempotency;
import org.springframework.data.redis.core.StringRedisTemplate;
import org.springframework.stereotype.Service;
import java.time.Duration;
import java.util.UUID;
@Service
public class CouponRedemptionService {
private final StringRedisTemplate redis;
public CouponRedemptionService(StringRedisTemplate redis) {
this.redis = redis;
}
public String redeemCoupon(String couponId, String accountId) {
String lockKey = "coupon-lock:" + couponId;
String token = UUID.randomUUID().toString();
// Acquire lock: SET NX PX ensures only one caller wins
Boolean acquired = redis.opsForValue()
.setIfAbsent(lockKey, token, Duration.ofSeconds(10));
if (Boolean.FALSE.equals(acquired)) {
System.out.println("Coupon " + couponId + " already claimed by another user");
return "ALREADY_CLAIMED";
}
try {
// Critical section: redeem the one-time coupon
System.out.println("Coupon " + couponId + " redeemed by " + accountId);
return "REDEEMED";
} finally {
// Release lock (check token to avoid releasing someone else's lock)
String current = redis.opsForValue().get(lockKey);
if (token.equals(current)) {
redis.delete(lockKey);
}
}
}
}
Expected output (first caller):
Coupon DIWALI50 redeemed by SAV2001
Expected output (second caller, concurrent):
Coupon DIWALI50 already claimed by another user
How do you choose between these techniques?
| Technique | Best For | Strength | Weakness |
|---|---|---|---|
| Idempotency key | Payments, commands | Replays stored response | Needs key store (Redis) |
| Unique constraint | Duplicate record prevention | Database-enforced, survives crashes | Only prevents duplicate rows |
| State-based check | State transitions | No extra storage needed | Needs atomic check-and-act |
| Distributed lock | One-time scarce resource | Works across instances | TTL hazards, latency |
package com.bank.idempotency;
public class TechniqueChoice {
public static void main(String[] args) {
System.out.println("Decision guide:");
System.out.println(" Replaying a response? -> Idempotency key");
System.out.println(" Preventing duplicate rows? -> Unique constraint");
System.out.println(" Skipping a completed transition? -> State check");
System.out.println(" Claiming a non-DB scarce resource? -> Distributed lock");
System.out.println();
System.out.println("Preference: DB constraint > state check > idem key > lock");
System.out.println("Reason: DB guarantees survive crashes; locks have TTL hazards");
}
}
Expected output:
Decision guide:
Replaying a response? -> Idempotency key
Preventing duplicate rows? -> Unique constraint
Skipping a completed transition? -> State check
Claiming a non-DB scarce resource? -> Distributed lock
Preference: DB constraint > state check > idem key > lock
Reason: DB guarantees survive crashes; locks have TTL hazards
How do these techniques compose?
A payment endpoint often uses an idempotency key at the edge AND a unique constraint in the database as a second line of defense.
package com.bank.idempotency;
public class ComposedIdempotency {
public static void main(String[] args) {
System.out.println("Composed defense for payment:");
System.out.println(" Layer 1: Idempotency key in Redis (fast, returns stored response)");
System.out.println(" Layer 2: UNIQUE(transaction_reference) in DB (hard backstop)");
System.out.println();
System.out.println("If Redis is down or key expired:");
System.out.println(" Layer 2 catches the duplicate at the database level");
System.out.println(" DuplicateKeyException -> return 'already processed'");
}
}
Expected output:
Composed defense for payment:
Layer 1: Idempotency key in Redis (fast, returns stored response)
Layer 2: UNIQUE(transaction_reference) in DB (hard backstop)
If Redis is down or key expired:
Layer 2 catches the duplicate at the database level
DuplicateKeyException -> return 'already processed'
What are the concurrency subtleties?
Two requests with the same idempotency key may arrive simultaneously. The store-and-check must be atomic.
package com.bank.idempotency;
public class ConcurrencyHandling {
// WRONG: read-then-write race
// Thread A: reads key -> not found
// Thread B: reads key -> not found
// Thread A: processes + stores
// Thread B: processes + stores -> DOUBLE CHARGE
// CORRECT: atomic claim with SET NX or INSERT with unique constraint
// Thread A: SET NX key -> success (claimed)
// Thread B: SET NX key -> failure (someone else claimed)
// Thread B: waits or returns 409
public static void main(String[] args) {
System.out.println("WRONG: read-then-write race -> double charge");
System.out.println("CORRECT: atomic SET NX or unique INSERT -> single winner");
System.out.println();
System.out.println("Handle in-progress case:");
System.out.println(" If key exists with IN_PROGRESS status:");
System.out.println(" Option A: wait and poll for result");
System.out.println(" Option B: return 409 Conflict, client retries later");
}
}
Expected output:
WRONG: read-then-write race -> double charge
CORRECT: atomic SET NX or unique INSERT -> single winner
Handle in-progress case:
If key exists with IN_PROGRESS status:
Option A: wait and poll for result
Option B: return 409 Conflict, client retries later
What should a senior engineer know?
Prefer database-enforced guards over distributed locks wherever possible, because unique constraints survive crashes while locks have TTL and expiry hazards.
Senior guidelines:
- Unique constraint: cheapest, survives crashes, no extra infra
- State check: no storage cost, needs atomic conditional update
- Idempotency key: needed when you must replay the full response
- Distributed lock: last resort when resource is not a DB row
- Always handle concurrent same-key: atomic claim, not read-then-write
- Compose: idempotency key at edge + unique constraint in DB
- Clean up: TTL on idempotency keys, periodic purge of processed messages
Visual Overview
Four Idempotency Techniques:
1. Idempotency Key (Payments)
┌────────┐ key: IK-001 ┌───────────┐ stored? ┌──────────┐
│ Client │──────────────▶│ Redis │──────────▶│ Process │
└────────┘ │ Key Store│ no ──────│ + Store │
│ │ yes ─────│ Replay │
└───────────┘ └──────────┘
2. Unique Constraint (Duplicate Prevention)
┌────────┐ INSERT ┌───────────┐
│Service │─────────▶│ Database │ UNIQUE(txn_ref)
└────────┘ │ │ duplicate -> reject
└───────────┘
3. State Check (Transitions)
┌────────┐ close() ┌───────────┐
│Service │──────────▶│ Account │ status=CLOSED?
└────────┘ │ │ yes -> skip
│ │ no -> close
└───────────┘
4. Distributed Lock (Scarce Resource)
┌────────┐ SET NX ┌───────────┐
│Service │─────────▶│ Redis │ acquired?
└────────┘ │ Lock │ yes -> claim resource
│ │ no -> already claimed
└───────────┘
Key Takeaway
There are four main ways to implement idempotency, and the skill is matching the technique to the scenario. Idempotency keys replay a stored response for payment retries. Unique constraints let the database reject duplicate records atomically. State-based validation skips an operation if the resource is already in the target state. Distributed locks serialize access for a scarce resource that is not a database row.
The preference order matters: unique constraints are cheapest and survive crashes, state checks need no extra storage, idempotency keys are needed when you must replay the full response, and distributed locks are a last resort because of TTL hazards. The techniques also compose: at National Bank of India, the payment endpoint uses an idempotency key in Redis at the edge for fast replay, plus a unique constraint on the transaction reference in the database as a hard backstop.
The concurrency subtlety that trips up many candidates is that the store-and-check must be atomic. A read-then-write sequence races under concurrent requests, so two threads both see "not found" and both process, causing a double charge. The fix is an atomic claim: SET NX in Redis or INSERT with a unique constraint, so exactly one caller wins.
Remember one sentence: match the idempotency technique to the scenario: idempotency key for response replay, unique constraint for duplicate prevention, state check for transitions, distributed lock for non-DB scarce resources, and always prefer database-enforced guards over locks.
---