Back to Resilience: Circuit Breaker, Retry, Bulkhead
Q8Question 8 of 9

What is a circuit breaker and how do you implement it in Spring Boot

Answer: A circuit breaker wraps calls to a downstream service, tracks failures in a sliding window, and automatically stops making calls (opens the circuit) when the failure rate exceeds a threshold, returning a fallback response instead of waiting for timeouts.

Key Concepts
  • A circuit breaker monitors calls to a downstream service and stops making calls when the failure rate exceeds a threshold, preventing cascading failures.
  • The circuit breaker has three states: CLOSED (normal, requests pass through), OPEN (requests are blocked, fallback is returned), and HALF_OPEN (limited requests are allowed to test if the service has recovered).
  • Resilience4j is the recommended circuit breaker library for Spring Boot; it provides @CircuitBreaker, @Retry, @RateLimiter, @Bulkhead, and @TimeLimiter annotations.
  • The failureRateThreshold (default 50%) determines when the circuit opens; slidingWindowSize (default 100) defines how many recent calls are evaluated.
  • waitDurationInOpenState (default 60 seconds) is how long the circuit stays open before transitioning to half-open and allowing test requests.
  • A fallback method runs when the circuit is open or the call fails, returning a default response instead of an error to the caller.

Definition

A circuit breaker wraps calls to a downstream service, tracks failures in a sliding window, and automatically stops making calls (opens the circuit) when the failure rate exceeds a threshold, returning a fallback response instead of waiting for timeouts.
  • Circuit breaker is a resilience pattern that prevents an application from repeatedly calling a failing service; it stops requests when failures exceed a threshold and resumes after a recovery period.
  • CLOSED state is the normal state where all requests pass through to the downstream service and results (success or failure) are recorded in a sliding window.
  • OPEN state is the tripped state where all requests are immediately rejected with a fallback response, without calling the downstream service at all.
  • HALF_OPEN state is the testing state where a limited number of requests are allowed through to check if the downstream service has recovered; if they succeed, the circuit closes, if they fail, it reopens.
  • Resilience4j is the lightweight fault tolerance library for Java that provides circuit breaker, retry, rate limiter, bulkhead, and time limiter as composable decorators.
  • Fallback method is the method that executes when the circuit is open or the call fails; it returns a default or cached response to the caller instead of an error.

Think of the National Bank of India ATM network. Normally, when Priya inserts her card, the ATM calls the central server to verify her account (CLOSED state). If the central server starts failing (network issue, server overload), the ATM notices that 5 out of the last 10 attempts failed (failureRateThreshold). The ATM puts up an "out of service" sign and stops trying to reach the server (OPEN state). After 60 seconds, the ATM tries one test transaction (HALF_OPEN state). If the test succeeds, the ATM goes back to normal (CLOSED). If it fails, the ATM stays out of service for another 60 seconds. Meanwhile, the "out of service" sign tells customers to use another ATM (fallback).

Understanding the Concepts

What problem does the circuit breaker solve?

In a microservices architecture, services depend on each other. When Service A calls Service B and Service B is slow or down, Service A's threads are blocked waiting for a response. If Service A has 200 threads and each one is waiting 30 seconds for Service B, all 200 threads are consumed within minutes. Service A becomes unresponsive, and services that depend on A fail too. This is a cascading failure.

The circuit breaker stops this cascade by failing fast. Instead of waiting 30 seconds for a timeout, the circuit breaker immediately returns a fallback response when it detects that Service B is failing.

JAVA
package com.bank;

public class CascadingFailureDemo {
    public static void main(String[] args) {
        System.out.println("Cascading failure without circuit breaker:");
        System.out.println();
        System.out.println("1. Fraud Check Service goes down");
        System.out.println("2. Payment Service calls Fraud Check, waits 30s timeout");
        System.out.println("3. Payment Service threads fill up (200 threads blocked)");
        System.out.println("4. Payment Service becomes unresponsive");
        System.out.println("5. Account Service calls Payment Service, waits 30s timeout");
        System.out.println("6. Account Service threads fill up");
        System.out.println("7. Mobile app shows error to all customers");
        System.out.println();
        System.out.println("Total time to full outage: ~2 minutes");
        System.out.println("Root cause: one service down, entire system down");
        System.out.println();
        System.out.println("With circuit breaker:");
        System.out.println("1. Fraud Check Service goes down");
        System.out.println("2. Circuit breaker detects 50% failure rate");
        System.out.println("3. Circuit OPENS: returns fallback in <1ms");
        System.out.println("4. Payment Service stays healthy");
        System.out.println("5. Account Service stays healthy");
        System.out.println("6. Mobile app works (payments flagged for manual review)");
    }
}
TEXT
Expected output:
Cascading failure without circuit breaker:

1. Fraud Check Service goes down
2. Payment Service calls Fraud Check, waits 30s timeout
3. Payment Service threads fill up (200 threads blocked)
4. Payment Service becomes unresponsive
5. Account Service calls Payment Service, waits 30s timeout
6. Account Service threads fill up
7. Mobile app shows error to all customers

Total time to full outage: ~2 minutes
Root cause: one service down, entire system down

With circuit breaker:
1. Fraud Check Service goes down
2. Circuit breaker detects 50% failure rate
3. Circuit OPENS: returns fallback in <1ms
4. Payment Service stays healthy
5. Account Service stays healthy
6. Mobile app works (payments flagged for manual review)

What are the three states of a circuit breaker?

The circuit breaker transitions between three states based on the success and failure rates of recent calls.

StateBehaviorTransition
CLOSEDAll requests pass through, results are recordedOpens when failure rate exceeds threshold
OPENAll requests are rejected immediately, fallback is returnedTransitions to HALF_OPEN after wait duration
HALF_OPENA limited number of test requests are allowed throughCloses if tests succeed, reopens if tests fail
JAVA
package com.bank;

public class CircuitBreakerStates {
    public static void main(String[] args) {
        System.out.println("Circuit Breaker State Machine:");
        System.out.println();
        System.out.println("CLOSED (normal operation)");
        System.out.println("  - All requests go to downstream service");
        System.out.println("  - Sliding window records last 10 calls");
        System.out.println("  - Failure rate calculated: failures / total");
        System.out.println("  - If failure rate > 50%: transition to OPEN");
        System.out.println();
        System.out.println("OPEN (circuit tripped)");
        System.out.println("  - All requests immediately return fallback");
        System.out.println("  - No calls to downstream service");
        System.out.println("  - Wait 60 seconds");
        System.out.println("  - After 60s: transition to HALF_OPEN");
        System.out.println();
        System.out.println("HALF_OPEN (testing recovery)");
        System.out.println("  - Allow 3 test requests through");
        System.out.println("  - If all 3 succeed: transition to CLOSED");
        System.out.println("  - If any fail: transition back to OPEN");
    }
}
TEXT
Expected output:
Circuit Breaker State Machine:

CLOSED (normal operation)
  - All requests go to downstream service
  - Sliding window records last 10 calls
  - Failure rate calculated: failures / total
  - If failure rate > 50%: transition to OPEN

OPEN (circuit tripped)
  - All requests immediately return fallback
  - No calls to downstream service
  - Wait 60 seconds
  - After 60s: transition to HALF_OPEN

HALF_OPEN (testing recovery)
  - Allow 3 test requests through
  - If all 3 succeed: transition to CLOSED
  - If any fail: transition back to OPEN

How do you configure Resilience4j in Spring Boot?

Resilience4j integrates with Spring Boot through application.properties or application.yml. You define named circuit breaker instances with their own configuration.

PROPERTIES
# application.properties

# Circuit breaker for Fraud Check Service
resilience4j.circuitbreaker.instances.fraudCheck.failure-rate-threshold=50
resilience4j.circuitbreaker.instances.fraudCheck.sliding-window-size=10
resilience4j.circuitbreaker.instances.fraudCheck.sliding-window-type=COUNT_BASED
resilience4j.circuitbreaker.instances.fraudCheck.minimum-number-of-calls=5
resilience4j.circuitbreaker.instances.fraudCheck.wait-duration-in-open-state=60s
resilience4j.circuitbreaker.instances.fraudCheck.permitted-number-of-calls-in-half-open-state=3
resilience4j.circuitbreaker.instances.fraudCheck.automatic-transition-from-open-to-half-open-enabled=true

# Circuit breaker for Notification Service (less critical, higher threshold)
resilience4j.circuitbreaker.instances.notification.failure-rate-threshold=70
resilience4j.circuitbreaker.instances.notification.sliding-window-size=20
resilience4j.circuitbreaker.instances.notification.wait-duration-in-open-state=30s
JAVA
package com.bank;

public class Resilience4jConfigExplained {
    public static void main(String[] args) {
        System.out.println("Resilience4j configuration for National Bank:");
        System.out.println();
        System.out.println("fraudCheck circuit breaker:");
        System.out.println("  failure-rate-threshold=50");
        System.out.println("    Open circuit if 50% of calls fail");
        System.out.println("  sliding-window-size=10");
        System.out.println("    Evaluate last 10 calls");
        System.out.println("  minimum-number-of-calls=5");
        System.out.println("    Do not evaluate until at least 5 calls recorded");
        System.out.println("  wait-duration-in-open-state=60s");
        System.out.println("    Stay open for 60 seconds before testing");
        System.out.println("  permitted-number-of-calls-in-half-open-state=3");
        System.out.println("    Allow 3 test calls in HALF_OPEN");
    }
}
TEXT
Expected output:
Resilience4j configuration for National Bank:

fraudCheck circuit breaker:
  failure-rate-threshold=50
    Open circuit if 50% of calls fail
  sliding-window-size=10
    Evaluate last 10 calls
  minimum-number-of-calls=5
    Do not evaluate until at least 5 calls recorded
  wait-duration-in-open-state=60s
    Stay open for 60 seconds before testing
  permitted-number-of-calls-in-half-open-state=3
    Allow 3 test calls in HALF_OPEN

How do you implement a circuit breaker with @CircuitBreaker annotation?

Resilience4j provides the @CircuitBreaker annotation that you place on a method. When the circuit opens, the fallback method is called instead.

JAVA
package com.bank.service;

import com.bank.dto.FraudCheckResult;
import io.github.resilience4j.circuitbreaker.annotation.CircuitBreaker;
import org.springframework.stereotype.Service;
import org.springframework.web.client.RestClient;

import java.math.BigDecimal;

@Service
public class FraudCheckService {

    private final RestClient restClient;

    public FraudCheckService(RestClient.Builder builder) {
        this.restClient = builder
            .baseUrl("http://fraud-service:8081")
            .build();
    }

    @CircuitBreaker(name = "fraudCheck", fallbackMethod = "fraudCheckFallback")
    public FraudCheckResult checkFraud(String accountNumber, BigDecimal amount) {
        System.out.println("  Calling Fraud Check Service for " + accountNumber);
        return restClient.get()
            .uri("/api/fraud/check?account={account}&amount={amount}", accountNumber, amount)
            .retrieve()
            .body(FraudCheckResult.class);
    }

    // Fallback: same parameters + Throwable
    private FraudCheckResult fraudCheckFallback(String accountNumber, BigDecimal amount, Throwable ex) {
        System.out.println("  FALLBACK: Fraud Check unavailable (" + ex.getMessage() + ")");
        System.out.println("  Returning: PENDING (manual review required)");
        return new FraudCheckResult(accountNumber, "PENDING", "Service unavailable, flagged for manual review");
    }
}
JAVA
package com.bank.dto;

// Java 25 record for fraud check result
public record FraudCheckResult(String accountNumber, String status, String reason) {}
TEXT
Expected output:
When Fraud Check Service is UP (circuit CLOSED):
  Calling Fraud Check Service for SAV2001
  Result: {accountNumber: SAV2001, status: CLEAN, reason: No fraud detected}

When Fraud Check Service is DOWN (circuit OPEN):
  FALLBACK: Fraud Check unavailable (Connection refused)
  Returning: PENDING (manual review required)
  Result: {accountNumber: SAV2001, status: PENDING, reason: Service unavailable, flagged for manual review}

What is the difference between circuit breaker and retry?

FeatureCircuit BreakerRetry
PurposeStop calling a failing serviceRe-attempt a failed call
When to useService is persistently downTransient failure (network blip)
Behavior on failureReturns fallback immediatelyWaits and tries again
State managementTracks failure rate over timeNo state between requests
Thread usageFrees threads immediatelyHolds thread during retries
ConfigurationThreshold, window, wait durationMax attempts, wait interval
Resilience4j annotation@CircuitBreaker@Retry

You can combine both: retry first (for transient failures), and if retries are exhausted, the circuit breaker records the failure.

JAVA
package com.bank.service;

import com.bank.dto.FraudCheckResult;
import io.github.resilience4j.circuitbreaker.annotation.CircuitBreaker;
import io.github.resilience4j.retry.annotation.Retry;
import org.springframework.stereotype.Service;
import org.springframework.web.client.RestClient;

import java.math.BigDecimal;

@Service
public class ResilientFraudCheckService {

    private final RestClient restClient;

    public ResilientFraudCheckService(RestClient.Builder builder) {
        this.restClient = builder.baseUrl("http://fraud-service:8081").build();
    }

    // Retry runs first (inner), circuit breaker wraps retry (outer)
    @CircuitBreaker(name = "fraudCheck", fallbackMethod = "fallback")
    @Retry(name = "fraudCheck")
    public FraudCheckResult checkFraud(String accountNumber, BigDecimal amount) {
        System.out.println("  Calling Fraud Check Service (may retry)");
        return restClient.get()
            .uri("/api/fraud/check?account={a}&amount={amt}", accountNumber, amount)
            .retrieve()
            .body(FraudCheckResult.class);
    }

    private FraudCheckResult fallback(String accountNumber, BigDecimal amount, Throwable ex) {
        System.out.println("  FALLBACK after retries exhausted: " + ex.getMessage());
        return new FraudCheckResult(accountNumber, "PENDING", "Manual review required");
    }
}
TEXT
Expected output (transient failure, retry succeeds):
  Calling Fraud Check Service (may retry)
  Calling Fraud Check Service (may retry)  [retry 1]
  Result: {status: CLEAN}

Expected output (persistent failure, retries exhausted, circuit opens):
  Calling Fraud Check Service (may retry)
  Calling Fraud Check Service (may retry)  [retry 1]
  Calling Fraud Check Service (may retry)  [retry 2]
  FALLBACK after retries exhausted: Connection refused
  Result: {status: PENDING}

What is a complete banking project with Resilience4j circuit breaker?

Project structure

DIAGRAM
payment-service/
├── pom.xml
└── src/
    ├── main/
    │   ├── java/
    │   │   └── com/
    │   │       └── bank/
    │   │           ├── PaymentServiceApp.java
    │   │           ├── controller/
    │   │           │   └── PaymentController.java
    │   │           ├── dto/
    │   │           │   ├── FraudCheckResult.java
    │   │           │   ├── PaymentRequest.java
    │   │           │   └── PaymentResponse.java
    │   │           └── service/
    │   │               ├── FraudCheckService.java
    │   │               └── PaymentService.java
    │   └── resources/
    │       └── application.properties
    └── test/
        └── java/
            └── com/
                └── bank/
                    └── service/
                        └── FraudCheckServiceTest.java

pom.xml

XML
<?xml version="1.0" encoding="UTF-8"?syn-kw">>
syn-kw"><project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
         http://maven.apache.org/xsd/maven-4.0.0.xsd"syn-kw">>
    syn-kw"><modelVersionsyn-kw">>4.0.0syn-kw"></modelVersionsyn-kw">>

    syn-kw"><parentsyn-kw">>
        syn-kw"><groupIdsyn-kw">>org.springframework.bootsyn-kw"></groupIdsyn-kw">>
        syn-kw"><artifactIdsyn-kw">>spring-boot-starter-parentsyn-kw"></artifactIdsyn-kw">>
        syn-kw"><versionsyn-kw">>3.4.7syn-kw"></versionsyn-kw">>
    syn-kw"></parentsyn-kw">>

    syn-kw"><groupIdsyn-kw">>com.banksyn-kw"></groupIdsyn-kw">>
    syn-kw"><artifactIdsyn-kw">>payment-servicesyn-kw"></artifactIdsyn-kw">>
    syn-kw"><versionsyn-kw">>1.0.0syn-kw"></versionsyn-kw">>

    syn-kw"><propertiessyn-kw">>
        syn-kw"><java.versionsyn-kw">>25syn-kw"></java.versionsyn-kw">>
    syn-kw"></propertiessyn-kw">>

    syn-kw"><dependenciessyn-kw">>
        syn-kw"><dependencysyn-kw">>
            syn-kw"><groupIdsyn-kw">>org.springframework.bootsyn-kw"></groupIdsyn-kw">>
            syn-kw"><artifactIdsyn-kw">>spring-boot-starter-websyn-kw"></artifactIdsyn-kw">>
        syn-kw"></dependencysyn-kw">>
        syn-kw"><dependencysyn-kw">>
            syn-kw"><groupIdsyn-kw">>org.springframework.bootsyn-kw"></groupIdsyn-kw">>
            syn-kw"><artifactIdsyn-kw">>spring-boot-starter-aopsyn-kw"></artifactIdsyn-kw">>
        syn-kw"></dependencysyn-kw">>
        syn-kw"><dependencysyn-kw">>
            syn-kw"><groupIdsyn-kw">>org.springframework.bootsyn-kw"></groupIdsyn-kw">>
            syn-kw"><artifactIdsyn-kw">>spring-boot-starter-actuatorsyn-kw"></artifactIdsyn-kw">>
        syn-kw"></dependencysyn-kw">>
        syn-kw"><dependencysyn-kw">>
            syn-kw"><groupIdsyn-kw">>io.github.resilience4jsyn-kw"></groupIdsyn-kw">>
            syn-kw"><artifactIdsyn-kw">>resilience4j-spring-boot3syn-kw"></artifactIdsyn-kw">>
            syn-kw"><versionsyn-kw">>2.3.0syn-kw"></versionsyn-kw">>
        syn-kw"></dependencysyn-kw">>
        syn-kw"><dependencysyn-kw">>
            syn-kw"><groupIdsyn-kw">>org.springframework.bootsyn-kw"></groupIdsyn-kw">>
            syn-kw"><artifactIdsyn-kw">>spring-boot-starter-testsyn-kw"></artifactIdsyn-kw">>
            syn-kw"><scopesyn-kw">>testsyn-kw"></scopesyn-kw">>
        syn-kw"></dependencysyn-kw">>
    syn-kw"></dependenciessyn-kw">>
syn-kw"></projectsyn-kw">>

DTOs (Java 25 records)

JAVA
package com.bank.dto;

import java.math.BigDecimal;

public record PaymentRequest(String fromAccount, String toAccount, BigDecimal amount) {}
JAVA
package com.bank.dto;

import java.math.BigDecimal;

public record PaymentResponse(String fromAccount, String toAccount, BigDecimal amount,
                               String status, String fraudCheckStatus) {}

PaymentService.java

JAVA
package com.bank.service;

import com.bank.dto.PaymentRequest;
import com.bank.dto.PaymentResponse;
import org.springframework.stereotype.Service;

@Service
public class PaymentService {

    private final FraudCheckService fraudCheckService;

    public PaymentService(FraudCheckService fraudCheckService) {
        this.fraudCheckService = fraudCheckService;
    }

    public PaymentResponse processPayment(PaymentRequest request) {
        System.out.println("Processing payment: " + request.amount() +
            " from " + request.fromAccount() + " to " + request.toAccount());

        // Call fraud check with circuit breaker
        var fraudResult = fraudCheckService.checkFraud(
            request.fromAccount(), request.amount());

        String status = switch (fraudResult.status()) {
            case "CLEAN" -> "APPROVED";
            case "FRAUD" -> "REJECTED";
            case "PENDING" -> "PENDING_REVIEW";
            default -> "UNKNOWN";
        };

        System.out.println("Payment status: " + status);
        return new PaymentResponse(
            request.fromAccount(),
            request.toAccount(),
            request.amount(),
            status,
            fraudResult.status()
        );
    }
}

PaymentController.java

JAVA
package com.bank.controller;

import com.bank.dto.PaymentRequest;
import com.bank.dto.PaymentResponse;
import com.bank.service.PaymentService;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/api/payments")
public class PaymentController {

    private final PaymentService paymentService;

    public PaymentController(PaymentService paymentService) {
        this.paymentService = paymentService;
    }

    @PostMapping
    public ResponseEntity<PaymentResponse> pay(@RequestBody PaymentRequest request) {
        var response = paymentService.processPayment(request);
        return ResponseEntity.ok(response);
    }
}

How to test

BASH
# Step 1: Start the application
cd payment-service
mvn spring-boot:run

# Step 2: Send a payment (fraud check service is down)
curl -X POST http://localhost:8080/api/payments \
  -H "Content-Type: application/json" \
  -d '{"fromAccount": "SAV2001", "toAccount": "SAV2002", "amount": 25000}'

# Step 3: Check circuit breaker state via actuator
curl http://localhost:8080/actuator/circuitbreakers
TEXT
Expected output:
# Payment with fraud check service DOWN:
Processing payment: 25000 from SAV2001 to SAV2002
  FALLBACK: Fraud Check unavailable (Connection refused)
  Returning: PENDING (manual review required)
Payment status: PENDING_REVIEW
{"fromAccount":"SAV2001","toAccount":"SAV2002","amount":25000,"status":"PENDING_REVIEW","fraudCheckStatus":"PENDING"}

# Actuator circuit breaker state:
{"circuitBreakers":{"fraudCheck":{"state":"OPEN","failureRate":100.0,"numberOfFailedCalls":5}}}

What are common mistakes with circuit breakers?

Wrong: circuit breaker with no fallback

JAVA
package com.bank.service;

import io.github.resilience4j.circuitbreaker.annotation.CircuitBreaker;
import org.springframework.stereotype.Service;
import org.springframework.web.client.RestClient;

@Service
public class NoFallbackService {

    private final RestClient restClient;

    public NoFallbackService(RestClient.Builder builder) {
        this.restClient = builder.baseUrl("http://fraud-service:8081").build();
    }

    // WRONG: no fallback method defined
    @CircuitBreaker(name = "fraudCheck")
    public String checkFraud(String accountNumber) {
        return restClient.get()
            .uri("/api/fraud/check?account={a}", accountNumber)
            .retrieve()
            .body(String.class);
    }
}
TEXT
Expected output:
When circuit is OPEN without fallback:
  io.github.resilience4j.circuitbreaker.CallNotPermittedException:
    CircuitBreaker 'fraudCheck' is OPEN and does not permit further calls

Problem: the exception propagates to the caller (500 Internal Server Error)
The whole point of a circuit breaker is to return a graceful fallback.

Correct: always define a fallback

JAVA
package com.bank.service;

import com.bank.dto.FraudCheckResult;
import io.github.resilience4j.circuitbreaker.annotation.CircuitBreaker;
import org.springframework.stereotype.Service;
import org.springframework.web.client.RestClient;

import java.math.BigDecimal;

@Service
public class WithFallbackService {

    private final RestClient restClient;

    public WithFallbackService(RestClient.Builder builder) {
        this.restClient = builder.baseUrl("http://fraud-service:8081").build();
    }

    // CORRECT: fallback method defined
    @CircuitBreaker(name = "fraudCheck", fallbackMethod = "fallback")
    public FraudCheckResult checkFraud(String accountNumber, BigDecimal amount) {
        return restClient.get()
            .uri("/api/fraud/check?account={a}&amount={amt}", accountNumber, amount)
            .retrieve()
            .body(FraudCheckResult.class);
    }

    private FraudCheckResult fallback(String accountNumber, BigDecimal amount, Throwable ex) {
        return new FraudCheckResult(accountNumber, "PENDING", "Flagged for manual review");
    }
}
TEXT
Expected output:
When circuit is OPEN with fallback:
  {accountNumber: SAV2001, status: PENDING, reason: Flagged for manual review}
  HTTP 200 OK (graceful degradation, no error to caller)

What should a senior or principal engineer know?

At the senior level, you should configure circuit breakers per downstream dependency, not a single global one. Each service has different failure characteristics, SLAs, and recovery times.

At the principal level, think about:

  • Observability: expose circuit breaker metrics through Actuator and Micrometer. Alert when any circuit breaker transitions to OPEN. Dashboard the failure rate, call count, and state for each circuit.
  • Bulkhead + circuit breaker: a bulkhead limits the number of concurrent calls to a service (e.g., max 10 threads for fraud check). This prevents one slow service from consuming all threads in the application.
  • Time limiter: set a maximum execution time for each call (e.g., 2 seconds for fraud check). If the call exceeds the limit, it is cancelled and counted as a failure by the circuit breaker.
  • Testing strategy: use WireMock to simulate downstream service failures in integration tests. Verify that the circuit opens after the threshold and that the fallback returns the correct response.
  • Fallback quality: a fallback should not just return null. Design meaningful degraded behavior: queue the request for later processing, return cached data, or return a partial response.
JAVA
package com.bank;

public class CircuitBreakerBestPractices {
    public static void main(String[] args) {
        System.out.println("Circuit Breaker Best Practices for National Bank:");
        System.out.println();
        System.out.println("1. One circuit breaker per downstream service");
        System.out.println("   fraudCheck, notificationService, exchangeRateService");
        System.out.println();
        System.out.println("2. Combine with retry: @Retry + @CircuitBreaker");
        System.out.println("   Retry handles transient failures");
        System.out.println("   Circuit breaker handles persistent failures");
        System.out.println();
        System.out.println("3. Combine with bulkhead: limit concurrent calls");
        System.out.println("   fraudCheck: max 10 concurrent, queue 20");
        System.out.println("   notification: max 5 concurrent, queue 10");
        System.out.println();
        System.out.println("4. Monitor: alert on circuit OPEN transitions");
        System.out.println("   Prometheus: resilience4j_circuitbreaker_state");
        System.out.println("   Grafana dashboard per circuit breaker");
        System.out.println();
        System.out.println("5. Meaningful fallbacks:");
        System.out.println("   Fraud check: PENDING (queue for manual review)");
        System.out.println("   Exchange rate: return last cached rate");
        System.out.println("   Notification: queue for retry later");
    }
}
TEXT
Expected output:
Circuit Breaker Best Practices for National Bank:

1. One circuit breaker per downstream service
   fraudCheck, notificationService, exchangeRateService

2. Combine with retry: @Retry + @CircuitBreaker
   Retry handles transient failures
   Circuit breaker handles persistent failures

3. Combine with bulkhead: limit concurrent calls
   fraudCheck: max 10 concurrent, queue 20
   notification: max 5 concurrent, queue 10

4. Monitor: alert on circuit OPEN transitions
   Prometheus: resilience4j_circuitbreaker_state
   Grafana dashboard per circuit breaker

5. Meaningful fallbacks:
   Fraud check: PENDING (queue for manual review)
   Exchange rate: return last cached rate
   Notification: queue for retry later

What are common interview follow-up questions?

  • What is the difference between COUNT_BASED and TIME_BASED sliding windows? (COUNT_BASED evaluates the last N calls regardless of time. TIME_BASED evaluates all calls within the last N seconds. COUNT_BASED is simpler and most commonly used.)
  • Can you use circuit breaker with WebClient (reactive)? (Yes. Resilience4j supports reactive types. Use the same @CircuitBreaker annotation or wrap the Mono/Flux with CircuitBreaker.decorateMono().)
  • What happens if the fallback method also fails? (The exception from the fallback propagates to the caller. Design fallback methods to be simple and unlikely to fail, like returning a static default.)
  • How does Netflix Hystrix compare to Resilience4j? (Hystrix is in maintenance mode since 2018. Resilience4j is the recommended replacement. It is lighter, modular, and designed for Java 8+ functional style. Spring Cloud Circuit Breaker uses Resilience4j as the default.)
  • Should you put the circuit breaker on the client or the server? (On the client. The client is the one making the call, and it decides how to handle failures. The server does not know about the client's resilience strategy.)
  • What is the minimum number of calls before the circuit evaluates? (Configured by minimumNumberOfCalls. Default is 100. If you set it to 5, the circuit will not evaluate the failure rate until at least 5 calls have been recorded in the sliding window.)

Visual Overview

DIAGRAM
 Circuit Breaker State Transitions
 ===================================

                    failure rate
                    > threshold
 ┌──────────┐  ─────────────────>  ┌──────────┐
 │  CLOSED  │                      │   OPEN   │
 │          │  <─────────────────  │          │
 │ Requests │   success rate >     │ Requests │
 │ pass     │   threshold in       │ blocked, │
 │ through  │   HALF_OPEN          │ fallback │
 └──────────┘                      └────┬─────┘
                                        │
                                        │ after waitDuration
                                        │
                                        ▼
                                   ┌──────────┐
                                   │ HALF_OPEN│
                                   │          │
                                   │ Limited  │
                                   │ test     │
                                   │ requests │
                                   └──────────┘
                                    │        │
                              tests pass   tests fail
                                    │        │
                                    ▼        ▼
                              ┌────────┐ ┌──────┐
                              │ CLOSED │ │ OPEN │
                              └────────┘ └──────┘


 Payment Service with Circuit Breaker
 =====================================

 ┌────────────┐     ┌─────────────────┐     ┌─────────────────┐
 │  Mobile    │     │ Payment Service │     │ Fraud Check     │
 │  App       │     │                 │     │ Service         │
 └─────┬──────┘     └────────┬────────┘     └────────┬────────┘
       │                     │                       │
       │  POST /pay          │                       │
       │ ──────────────────> │                       │
       │                     │ checkFraud()          │
       │                     │ ─── [Circuit CLOSED] ─│─> call service
       │                     │ <──────────────────── │
       │                     │                       │
       │  200 APPROVED       │                       │
       │ <────────────────── │                       │
       │                     │                       │
       │  POST /pay          │                       │ SERVICE DOWN
       │ ──────────────────> │                       │ XXXXXXXXXX
       │                     │ checkFraud()          │
       │                     │ ─── [Circuit OPEN] ── │ (NOT called)
       │                     │ -> fallback()         │
       │                     │                       │
       │  200 PENDING_REVIEW │                       │
       │ <────────────────── │                       │

Key Takeaway

The circuit breaker pattern is essential in microservices because one failing service can take down the entire system. Without a circuit breaker, when the fraud check service is down, every payment request in the payment service waits for a 30-second timeout, consuming threads until the payment service itself becomes unresponsive. The cascade continues to every service that depends on the payment service.

A circuit breaker monitors calls to a downstream service through a sliding window. When the failure rate exceeds a threshold (e.g., 50% of the last 10 calls), the circuit opens. In the open state, all calls immediately return a fallback response without contacting the downstream service. After a wait period (e.g., 60 seconds), the circuit transitions to half-open and allows a few test calls. If the tests succeed, the circuit closes and normal operation resumes. If they fail, the circuit reopens.

In our National Bank of India payment-service, we used Resilience4j's @CircuitBreaker annotation on the fraud check call. When the fraud check service went down, the circuit opened and returned a "PENDING" result, flagging the payment for manual review instead of failing the entire transaction. The payment service stayed healthy and responsive. We combined @CircuitBreaker with @Retry so transient failures (network blips) were retried first, and only persistent failures triggered the circuit.

At the senior level, configure separate circuit breakers per downstream service with appropriate thresholds. At the principal level, combine circuit breakers with bulkheads (to limit concurrent calls) and time limiters (to cap execution time), monitor all circuit states through Actuator and Prometheus, and design meaningful fallbacks that provide degraded but useful behavior.

Remember one sentence: a circuit breaker stops calling a failing service after the failure rate exceeds a threshold, returning a fallback response immediately to prevent cascading failures across the system.

---

Share
Use ← → arrow keys to navigate