What is the bulkhead pattern? | Concurrency & Multithreading

Q: What is wrong with a shared pool?

// WRONG: one giant shared pool ExecutorService sharedPool = Executors.newFixedThreadPool(60); sharedPool.submit(() callPaymentGateway()); // 1000 calls/s sharedPool.submit(() getUserProfile()); // 500 calls/s sharedPool.submit(() sendSMS()); // 200 calls/s // Payment gateway goes down: 30s timeout

Q: How does the bulkhead pattern fix this?

package com.bank.core; import java.util.concurrent.; import java.util.concurrent.atomic.AtomicInteger; public class BulkheadConfig { private static final AtomicInteger counter = new AtomicInteger(); public static ThreadPoolExecutor paymentPool() { return new ThreadPoolExecutor( 10, 20, 60, TimeUnit.

Q: How do you monitor each pool?

ThreadPoolExecutor pool = paymentPool(); System.out.println("Active: " + pool.getActiveCount()); System.out.println("Queue: " + pool.getQueue().size()); System.out.println("Completed: " + pool.getCompletedTaskCount()); // Alert: if queue 80% capacity, pool is saturating

Q: How do you configure bulkheads in Spring Boot?

// @Bean("paymentExecutor") // public Executor paymentExecutor() { // ThreadPoolTaskExecutor e = new ThreadPoolTaskExecutor(); // e.setCorePoolSize(10); e.setMaxPoolSize(20); // e.setQueueCapacity(100); // e.setThreadNamePrefix("payment"); // e.setRejectedExecutionHandler(new AbortPolicy()); // retu

Definition

The bulkhead pattern isolates different workloads into separate bounded thread pools, each with its own size and rejection policy, so that one exhausted pool does not cascade failure to others.

Bulkhead pattern – named after ship compartments; isolates workloads so one failure is contained.
Shared pool problem – one slow dependency exhausts all threads in the shared pool, causing cascading failure across all workloads.
Bounded pools – each workload gets its own pool with core/max size, bounded queue, and rejection policy.
Rejection policies – AbortPolicy (fail fast, 503), CallerRunsPolicy (back-pressure), DiscardOldestPolicy (drop stale work).
Independent monitoring – each pool's active count, queue size, and completed count are tracked separately.
Implementation – custom ThreadPoolExecutor per workload, or Resilience4j Bulkhead.

A ship has bulkhead walls dividing it into compartments. If one compartment floods, the walls contain the water and other compartments stay dry. The ship stays afloat. Without bulkheads: one hole floods the entire ship and it sinks. At National Bank of India: payment pool (20 threads), user pool (30 threads), notification pool (10 threads). Payment gateway down: 20 payment threads blocked. User requests and notifications: unaffected, their pools are separate.

Understanding the Concepts

What is wrong with a shared pool?

JAVA

// WRONG: one giant shared pool
ExecutorService sharedPool = Executors.newFixedThreadPool(60);
sharedPool.submit(() -> callPaymentGateway());  // 1000 calls/s
sharedPool.submit(() -> getUserProfile());       // 500 calls/s
sharedPool.submit(() -> sendSMS());              // 200 calls/s

// Payment gateway goes down: 30s timeout per call
// 60 threads x 30s timeout = ALL threads consumed
// getUserProfile: QUEUED behind payments
// sendSMS: QUEUED behind payments
// EVERYTHING IS DOWN!

How does the bulkhead pattern fix this?

JAVA

package com.bank.core;

import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicInteger;

public class BulkheadConfig {
    private static final AtomicInteger counter = new AtomicInteger();

    public static ThreadPoolExecutor paymentPool() {
        return new ThreadPoolExecutor(
            10, 20, 60, TimeUnit.SECONDS,
            new LinkedBlockingQueue<>(100),
            r -> new Thread(r, "payment-" + counter.incrementAndGet()),
            new ThreadPoolExecutor.AbortPolicy() // fail fast: 503
        );
    }

    public static ThreadPoolExecutor userPool() {
        return new ThreadPoolExecutor(
            20, 30, 60, TimeUnit.SECONDS,
            new LinkedBlockingQueue<>(200),
            r -> new Thread(r, "user-" + counter.incrementAndGet()),
            new ThreadPoolExecutor.CallerRunsPolicy() // back-pressure
        );
    }

    public static ThreadPoolExecutor notifyPool() {
        return new ThreadPoolExecutor(
            5, 10, 60, TimeUnit.SECONDS,
            new LinkedBlockingQueue<>(500),
            r -> new Thread(r, "notify-" + counter.incrementAndGet()),
            new ThreadPoolExecutor.DiscardOldestPolicy() // drop stale
        );
    }
}

Now when the payment gateway is down, only the payment pool is exhausted. User requests and notifications continue normally.

What rejection policies are available?

| Policy | Behavior | Use case |
|---|---|---|
| AbortPolicy | Throws RejectedExecutionException | Critical: fail fast, return 503 |
| CallerRunsPolicy | Caller thread runs the task | Natural back-pressure |
| DiscardPolicy | Silently drops the task | Non-critical, expendable |
| DiscardOldestPolicy | Drops oldest queued task | Non-critical, prefer fresh |

How do you monitor each pool?

JAVA

ThreadPoolExecutor pool = paymentPool();
System.out.println("Active: " + pool.getActiveCount());
System.out.println("Queue: " + pool.getQueue().size());
System.out.println("Completed: " + pool.getCompletedTaskCount());
// Alert: if queue > 80% capacity, pool is saturating

How do you configure bulkheads in Spring Boot?

JAVA

// @Bean("paymentExecutor")
// public Executor paymentExecutor() {
//     ThreadPoolTaskExecutor e = new ThreadPoolTaskExecutor();
//     e.setCorePoolSize(10); e.setMaxPoolSize(20);
//     e.setQueueCapacity(100);
//     e.setThreadNamePrefix("payment-");
//     e.setRejectedExecutionHandler(new AbortPolicy());
//     return e;
// }
// @Async("paymentExecutor") public CF<PaymentResult> process() { ... }

What about Resilience4j Bulkhead?

Resilience4j provides a Bulkhead abstraction that limits concurrent calls without managing a thread pool directly.

JAVA

// Bulkhead bulkhead = Bulkhead.of("payment",
//     BulkheadConfig.custom()
//         .maxConcurrentCalls(20)
//         .maxWaitDuration(Duration.ofMillis(500))
//         .build());

What are common interview follow-up questions?

How do you size each pool? Based on expected throughput and IO latency per downstream. Formula: maxConcurrent = throughput x latency.
What is the difference between thread pool bulkhead and semaphore bulkhead? Thread pool isolates execution; semaphore limits concurrency without allocating threads.
Should every microservice call have its own pool? Yes, at least for critical downstream dependencies.

Visual Overview

DIAGRAM

  Shared Pool (WRONG)              Bulkhead Pools (CORRECT)
  ┌──────────────────┐             ┌─────────────┐
  │ 60 threads shared│             │ Payment (20)│
  │ Payments ────────│             │ AbortPolicy │
  │ Users    ────────│             └─────────────┘
  │ Notify   ────────│             ┌─────────────┐
  │                  │             │ User (30)   │
  │ Gateway down:    │             │ CallerRuns  │
  │ ALL 60 blocked!  │             └─────────────┘
  │ Users DOWN!      │             ┌─────────────┐
  │ Notify DOWN!     │             │ Notify (10) │
  └──────────────────┘             │ DiscardOld  │
                                   └─────────────┘
                                   Gateway down:
                                   Payment: 503
                                   User: OK!
                                   Notify: OK!

Key Takeaway

The bulkhead pattern isolates different workloads into separate, bounded thread pools. When one pool is exhausted because a downstream dependency is slow or down, other pools continue operating normally. A single giant shared pool is the wrong design: one slow dependency exhausts all threads and everything fails.

Each pool is configured for its workload: core/max size based on expected throughput, bounded queue to prevent OOM, rejection policy for overflow behavior, named threads for debugging, and independent monitoring. AbortPolicy for critical paths (fail fast, return 503). CallerRunsPolicy for natural back-pressure. DiscardOldestPolicy for non-critical notifications.

At National Bank of India, a payment gateway timeout on the shared 100-thread pool meant 500 users could not log in. Splitting into paymentPool(20), userPool(50), and notifyPool(10) contained the gateway failure to payments only. Users logged in normally.

Remember one sentence: the bulkhead pattern isolates failures by giving each workload its own bounded pool.

---