Definition
- min.insync.replicas -- a broker and topic configuration that sets the minimum number of in-sync replicas that must acknowledge a write for it to succeed when the producer uses
acks=all. - In-Sync Replica set (ISR) -- the set of replicas fully caught up with the leader; when a follower falls behind, it leaves the ISR; when it catches up, it rejoins.
- acks=all -- the producer waits for every replica in the ISR to persist the record before the broker acknowledges.
- NotEnoughReplicas -- the error the broker returns when the ISR drops below min.insync.replicas during an acks=all write.
- Durability-availability trade-off -- a higher min.insync.replicas is safer but makes writes unavailable sooner when brokers fail.
- Standard banking config -- replication factor 3, min.insync.replicas 2, acks=all: tolerates one broker loss, halts writes on two.
Imagine National Bank of India requires that every important ledger entry be signed by at least two independent officers before it counts as recorded. Setting acks=all is like Rajesh saying I will only consider my transfer done once all on-duty officers have signed. Setting min.insync.replicas to two is the bank's rule that at least two officers must be on duty and signing, otherwise the transaction is refused rather than recorded by a single officer who might go home with the only copy.
Understanding the Concepts
What is the In-Sync Replica set (ISR)?
Each partition has one leader and several follower replicas. The ISR is the set of replicas fully caught up with the leader. When a follower falls too far behind, it is removed from the ISR. When it catches up, it rejoins.
package com.bank.replication;
public class IsrExplained {
public static void main(String[] args) {
System.out.println("Partition 0 (replication factor 3):");
System.out.println(" Broker 1: LEADER (ISR)");
System.out.println(" Broker 2: follower (ISR) -- caught up");
System.out.println(" Broker 3: follower (ISR) -- caught up");
System.out.println();
System.out.println("After Broker 3 falls behind:");
System.out.println(" Broker 1: LEADER (ISR)");
System.out.println(" Broker 2: follower (ISR)");
System.out.println(" Broker 3: follower (OUT of ISR)");
System.out.println(" ISR size: 2");
}
}
Partition 0 (replication factor 3):
Broker 1: LEADER (ISR)
Broker 2: follower (ISR) -- caught up
Broker 3: follower (ISR) -- caught up
After Broker 3 falls behind:
Broker 1: LEADER (ISR)
Broker 2: follower (ISR)
Broker 3: follower (OUT of ISR)
ISR size: 2
Why does acks=all alone have a gap?
With acks=all the producer waits for every replica in the ISR. But if the ISR shrinks to just the leader, acks=all means acks on one replica. If that leader then fails, an acknowledged record is lost.
package com.bank.replication;
public class AcksAllGap {
public static void main(String[] args) {
System.out.println("ISR = {Broker1, Broker2, Broker3}:");
System.out.println(" acks=all means write on 3 replicas -- SAFE");
System.out.println();
System.out.println("ISR shrinks to {Broker1} only:");
System.out.println(" acks=all means write on 1 replica");
System.out.println(" Broker1 fails -> acknowledged record LOST!");
System.out.println();
System.out.println("This is the gap min.insync.replicas closes.");
}
}
ISR = {Broker1, Broker2, Broker3}:
acks=all means write on 3 replicas -- SAFE
ISR shrinks to {Broker1} only:
acks=all means write on 1 replica
Broker1 fails -> acknowledged record LOST!
This is the gap min.insync.replicas closes.
How does min.insync.replicas close the gap?
min.insync.replicas tells the broker the minimum ISR size for an acks=all write. If the ISR drops below this number, the broker rejects the write with NotEnoughReplicas.
package com.bank.config;
public class MinIsrConfig {
public static void main(String[] args) {
System.out.println("Configuration:");
System.out.println(" replication.factor = 3");
System.out.println(" min.insync.replicas = 2");
System.out.println(" acks = all");
System.out.println();
System.out.println("ISR = 3: write succeeds (3 >= 2)");
System.out.println("ISR = 2: write succeeds (2 >= 2)");
System.out.println("ISR = 1: write REJECTED (1 < 2)");
System.out.println(" -> NotEnoughReplicasException");
System.out.println(" -> Producer retries until ISR recovers");
}
}
Configuration:
replication.factor = 3
min.insync.replicas = 2
acks = all
ISR = 3: write succeeds (3 >= 2)
ISR = 2: write succeeds (2 >= 2)
ISR = 1: write REJECTED (1 < 2)
-> NotEnoughReplicasException
-> Producer retries until ISR recovers
How do you set the configuration?
# Set at topic level
kafka-configs --alter --topic transfers \
--add-config min.insync.replicas=2
# Create topic with replication factor
kafka-topics --create --topic transfers \
--partitions 6 --replication-factor 3
package com.bank.producer;
import org.apache.kafka.clients.producer.ProducerConfig;
import java.util.Properties;
public class DurableProducerConfig {
public static Properties config() {
Properties props = new Properties();
props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "broker1:9092");
props.put(ProducerConfig.ACKS_CONFIG, "all");
props.put(ProducerConfig.ENABLE_IDEMPOTENCE_CONFIG, "true");
props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG,
"org.apache.kafka.common.serialization.StringSerializer");
props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG,
"org.apache.kafka.common.serialization.StringSerializer");
return props;
}
}
What is the durability-availability trade-off? (comparison)
| Config | Broker losses tolerated (writes continue) | Broker losses tolerated (no data loss) | Risk |
|---|---|---|---|
| RF=3, min.isr=1, acks=all | 2 | 0 (ISR can be 1) | Acknowledged data can be lost |
| RF=3, min.isr=2, acks=all | 1 | 1 | Standard safe config |
| RF=3, min.isr=3, acks=all | 0 | 2 | Any broker loss halts writes |
What are common mistakes?
package com.bank.mistake;
public class MinIsrMistakes {
public static void main(String[] args) {
// WRONG: acks=all without min.insync.replicas
System.out.println("WRONG: acks=all, min.insync.replicas=1 (default)");
System.out.println(" ISR can shrink to 1 -> acknowledged data at risk");
System.out.println();
// CORRECT: pair acks=all with min.insync.replicas=2
System.out.println("CORRECT: acks=all, min.insync.replicas=2, RF=3");
System.out.println();
// WRONG: min.insync.replicas = replication.factor
System.out.println("WRONG: min.insync.replicas=3, RF=3");
System.out.println(" One broker loss halts ALL writes -- too fragile");
System.out.println();
// WRONG: min.insync.replicas with acks=1
System.out.println("WRONG: min.insync.replicas=2 but acks=1");
System.out.println(" acks=1 never checks ISR size -- setting is useless");
}
}
WRONG: acks=all, min.insync.replicas=1 (default)
ISR can shrink to 1 -> acknowledged data at risk
CORRECT: acks=all, min.insync.replicas=2, RF=3
WRONG: min.insync.replicas=3, RF=3
One broker loss halts ALL writes -- too fragile
WRONG: min.insync.replicas=2 but acks=1
acks=1 never checks ISR size -- setting is useless
How do you monitor ISR health?
Monitoring the ISR set is critical because a shrinking ISR means replicas are falling behind and durability is weakening. Alert on UnderReplicatedPartitions and UnderMinIsrPartitionCount JMX metrics. A partition where the ISR set drops below min.insync.replicas will reject writes if acks=all, causing producer exceptions.
package com.bank.kafka;
public class IsrMonitoring {
public static void main(String[] args) {
System.out.println("ISR monitoring checklist:");
System.out.println(" JMX: kafka.server:type=ReplicaManager,name=UnderReplicatedPartitions");
System.out.println(" Alert if > 0 for more than 5 minutes");
System.out.println();
System.out.println(" JMX: kafka.server:type=ReplicaManager,name=UnderMinIsrPartitionCount");
System.out.println(" Alert immediately: writes will fail with acks=all");
System.out.println();
System.out.println(" CLI: kafka-topics.sh --describe --under-replicated-partitions");
System.out.println(" Shows which partitions have shrunk ISR");
System.out.println();
System.out.println(" Grafana dashboard: ISR size per partition over time");
}
}
Expected output:
ISR monitoring checklist:
JMX: kafka.server:type=ReplicaManager,name=UnderReplicatedPartitions
Alert if > 0 for more than 5 minutes
JMX: kafka.server:type=ReplicaManager,name=UnderMinIsrPartitionCount
Alert immediately: writes will fail with acks=all
CLI: kafka-topics.sh --describe --under-replicated-partitions
Shows which partitions have shrunk ISR
Grafana dashboard: ISR size per partition over time
What should a senior engineer know?
The two settings only deliver strong durability when used together. min.insync.replicas without acks=all is useless. Replication factor must be strictly greater than min.insync.replicas to tolerate a broker loss while still accepting writes. This is the deliberate choice to refuse data rather than risk losing it.
Visual Overview
min.insync.replicas + acks=all Replication Factor = 3, min.insync.replicas = 2 Normal (ISR = 3): ┌────────┐ ┌────────┐ ┌────────┐ │Broker 1│ │Broker 2│ │Broker 3│ │LEADER │ │follower│ │follower│ │ ISR │ │ ISR │ │ ISR │ └────────┘ └────────┘ └────────┘ Producer write -> all 3 ack -> SUCCESS One broker down (ISR = 2): ┌────────┐ ┌────────┐ ┌────────┐ │Broker 1│ │Broker 2│ │Broker 3│ │LEADER │ │follower│ │ DOWN │ │ ISR │ │ ISR │ │ │ └────────┘ └────────┘ └────────┘ ISR=2 >= min.isr=2 -> write SUCCESS Two brokers down (ISR = 1): ┌────────┐ ┌────────┐ ┌────────┐ │Broker 1│ │Broker 2│ │Broker 3│ │LEADER │ │ DOWN │ │ DOWN │ │ ISR │ │ │ │ │ └────────┘ └────────┘ └────────┘ ISR=1 < min.isr=2 -> write REJECTED NotEnoughReplicasException
Key Takeaway
min.insync.replicas is the setting that closes the gap in acks=all. With acks=all the producer waits for every replica in the ISR, but if the ISR shrinks to just the leader, acks=all means acks on one, and if that leader fails, an acknowledged record is lost. min.insync.replicas tells the broker the minimum ISR size required for an acks=all write, and if the ISR drops below that number, the broker rejects the write rather than risking data.
The standard safe banking configuration is replication factor 3, min.insync.replicas 2, acks=all. This tolerates one broker loss with no data loss and continued writes. Two broker losses halt writes rather than risking loss, which is the deliberate trade-off of durability over availability.
At National Bank of India, Rajesh once ran transfers with acks=all but min.insync.replicas of 1. During an incident two brokers failed, and when the leader also failed, acknowledged transfers were lost. Pankaj moved to RF=3, min.isr=2, acks=all, so now a single broker loss is tolerated and a two-broker loss halts writes to protect data.
Remember one sentence: acks=all is only as durable as the ISR is deep, and min.insync.replicas guarantees that depth by refusing writes when the ISR is too small.
---