Circuit Breaker pattern: Closed, Open, Half-Open states, tuning, pitfalls, and when it is harmful

• Circuit Breaker – a stability pattern that wraps a call to a fragile dependency and monitors it for failures.
• Closed state – calls flow normally; the breaker counts failures in a sliding window.
• Open state – failures exceeded the threshold; all calls are immediately rejected or routed to a fallback.
• Half-Open state – after a reset timeout, a few trial calls are allowed to test if the dependency recovered.
• Resilience4j – the most popular Java circuit-breaker library, configuring thresholds, windows, and wait durations.
• Purpose – protects both the caller and the struggling dependency from cascading failure.

What is the Circuit Breaker pattern?

The Circuit Breaker wraps a call to a fragile dependency and monitors it for failures. When failures exceed a threshold, it trips to Open and fails fast. After a reset timeout it moves to Half-Open to test recovery.

package com.bank.resilience;

import io.github.resilience4j.circuitbreaker.CircuitBreaker;
import io.github.resilience4j.circuitbreaker.CircuitBreakerConfig;
import java.math.BigDecimal;
import java.time.Duration;
import java.util.function.Supplier;

public class CreditScoreService {
    private final CircuitBreaker breaker;
    private final CibilApi cibilApi;

    public CreditScoreService(CibilApi cibilApi) {
        this.cibilApi = cibilApi;
        CircuitBreakerConfig config = CircuitBreakerConfig.custom()
            .failureRateThreshold(50)
            .slidingWindowSize(20)
            .waitDurationInOpenState(Duration.ofSeconds(10))
            .permittedNumberOfCallsInHalfOpenState(3)
            .build();
        this.breaker = CircuitBreaker.of("cibil", config);
    }

    public int getScore(String pan) {
        Supplier<Integer> decorated = CircuitBreaker
            .decorateSupplier(breaker, () -> cibilApi.fetch(pan));
        try {
            return decorated.get();
        } catch (Exception e) {
            return fallbackScore(); // conservative default
        }
    }

    private int fallbackScore() {
        return 600; // conservative internal rule
    }
}

How do the three states work?

In Closed state, calls flow normally and the breaker counts failures in a sliding window (count-based or time-based). When the failure rate crosses the threshold, the breaker transitions to Open. In Open, every call is immediately rejected with a CallNotPermittedException. After the wait duration, the breaker moves to Half-Open, allowing a limited number of trial calls. If those succeed above a success threshold, it returns to Closed; if they fail, it reopens.

// State transitions
// Closed: failure rate < 50% → stay Closed
// Closed: failure rate >= 50% → trip to Open
// Open: wait 10 seconds → move to Half-Open
// Half-Open: 3 trial calls succeed → back to Closed
// Half-Open: trial calls fail → back to Open

How do you tune the breaker?

The three key parameters are:

• Failure threshold: what percentage of failures triggers the trip. 50% over 20 calls is a common starting point.
• Call timeout: how long to wait before counting a call as failed. A slow call is often worse than a fast failure.
• Wait duration in open state: how long to stop hammering the sick service. Too short: you re-trip constantly. Too long: you stay down after recovery.

CircuitBreakerConfig config = CircuitBreakerConfig.custom()
    .failureRateThreshold(50)                          // 50% failure rate
    .slidingWindowSize(20)                             // last 20 calls
    .slowCallRateThreshold(80)                         // slow calls count too
    .slowCallDurationThreshold(Duration.ofSeconds(3))  // 3s = slow
    .waitDurationInOpenState(Duration.ofSeconds(10))   // 10s rest
    .permittedNumberOfCallsInHalfOpenState(3)          // 3 trial calls
    .build();

Can the breaker be a single point of failure?

If one shared breaker instance guards a resource for the whole fleet and has a bug, yes. The mitigation is to keep breaker state per-instance (each service node has its own breaker) and use a well-tested library like Resilience4j rather than hand-rolling.

// Per-instance breaker: each node tracks independently
// Node A: breaker trips because CIBIL is slow from this node
// Node B: breaker stays closed because its calls succeed
// No shared state means no single point of failure

How do you prevent cascading failures?

Combine the breaker with timeouts and bulkheads. If service A calls B calls C, and C is down, C's breaker trips and returns a fast fallback. This lets B's breaker stay healthy, which keeps A responsive.

// Bulkhead: isolate thread pool so one slow dependency
// cannot exhaust all threads
Bulkhead bulkhead = Bulkhead.of("cibil", BulkheadConfig.custom()
    .maxConcurrentCalls(10)
    .build());

// Breaker + bulkhead together
Supplier<Integer> decorated = Decorators.ofSupplier(() -> cibilApi.fetch(pan))
    .withCircuitBreaker(breaker)
    .withBulkhead(bulkhead)
    .decorate();

What is the fallback-depends-on-failing-service trap?

If your fallback for a failed balance lookup reads from a cache populated by the same failing service, the fallback is also stale or empty. The fallback must be genuinely independent.

// Wrong: fallback depends on the failing service
private int fallbackScore() {
    return cibilCache.get(pan); // populated by CIBIL, so also stale
}

// Correct: fallback is independent
private int fallbackScore() {
    return 600; // conservative internal rule, no CIBIL dependency
}

What are the pitfalls in async/Kafka systems?

In async systems there is no synchronous caller waiting. Tripping a breaker on a Kafka consumer means pausing consumption or routing to a retry topic or dead-letter queue, not throwing an exception.

// Kafka consumer with circuit breaker
try {
    processMessage(record);
} catch (Exception e) {
    if (breaker.getState() == CircuitBreaker.State.OPEN) {
        consumer.pause(partitions); // stop consuming
        sendToDeadLetterQueue(record);
    }
}

A common mistake is wrapping the producer send in a breaker but forgetting that messages accumulate in the topic, causing a thundering herd when the breaker closes.

When is a breaker harmful?

A breaker is harmful when it guards a call that is cheap, local, and reliable. It is also harmful when the fallback is worse than a brief delay, when the threshold is so sensitive it trips on normal blips, or when it hides a real bug that should be fixed.

// Harmful: breaker on a local in-memory cache (no network involved)
CircuitBreaker.decorateSupplier(breaker, () -> localCache.get(key));
// The breaker adds latency for no benefit

// Appropriate: breaker on an external HTTP call
CircuitBreaker.decorateSupplier(breaker, () -> externalApi.call());

What should a senior engineer know?

An adaptive breaker adjusts thresholds dynamically based on observed latency and error patterns. This is useful when traffic varies wildly but adds complexity. Most teams start with static thresholds and move to adaptive only with strong monitoring.

At National Bank of India, the loan approval service depended on the CIBIL API. During a CIBIL outage, every call hung for thirty seconds. Pankaj put a circuit breaker with a bulkhead around the call so that once CIBIL started failing, loan requests failed fast and fell back to a conservative internal rule, stopping one sick dependency from cascading into a full outage.