Definition
- Fork-join / scatter-gather – the parent splits work into independent child tasks and runs them concurrently.
- supplyAsync – submits each child to a thread pool, returning a
CompletableFutureimmediately. - allOf – waits for every child to complete; total time equals the slowest child.
- Timeout – bound the join so one slow dependency cannot hang the whole request.
- exceptionally – attach per-child fallbacks so a failed child contributes a default instead of crashing the aggregate.
- Dedicated executor – use a bounded pool for I/O children, not the shared
ForkJoinPool.
When Rajesh opens his portfolio dashboard at National Bank of India, the system needs three things: his account balance, his stock holdings value, and his mutual-fund value. Instead of fetching them one after another, the dashboard sends three runners out at the same time, one to each department. The dashboard waits until all three return, then adds the numbers into one total net worth. The three trips happen in parallel, so the dashboard loads in the time of the slowest trip, not the sum of all three.
Understanding the Concepts
What is the fork-join or scatter-gather pattern?
The execution flow has four phases: split, fork, join, and combine. The parent divides the work into genuinely independent units, runs them concurrently, waits for all to finish, and merges the results. Independence is essential: if child tasks depend on each other you cannot run them concurrently.
package com.bank.portfolio;
import java.math.BigDecimal;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
public class PortfolioService {
private final AccountService accountService;
private final EquityService equityService;
private final MutualFundService fundService;
private final ExecutorService executor;
public PortfolioService(AccountService accountService,
EquityService equityService,
MutualFundService fundService) {
this.accountService = accountService;
this.equityService = equityService;
this.fundService = fundService;
// Dedicated bounded executor for I/O calls
this.executor = Executors.newFixedThreadPool(10);
}
public BigDecimal getNetWorth(String customerId) throws Exception {
// FORK: submit all three children concurrently
CompletableFuture<BigDecimal> cashF =
CompletableFuture.supplyAsync(
() -> accountService.getTotalBalance(customerId),
executor);
CompletableFuture<BigDecimal> equityF =
CompletableFuture.supplyAsync(
() -> equityService.getHoldingsValue(customerId),
executor)
.exceptionally(ex -> BigDecimal.ZERO); // Fallback
CompletableFuture<BigDecimal> fundF =
CompletableFuture.supplyAsync(
() -> fundService.getHoldingsValue(customerId),
executor)
.exceptionally(ex -> BigDecimal.ZERO); // Fallback
// JOIN: wait for all with timeout
CompletableFuture.allOf(cashF, equityF, fundF)
.get(2, TimeUnit.SECONDS);
// COMBINE: merge results
BigDecimal cash = cashF.join();
BigDecimal equity = equityF.join();
BigDecimal fund = fundF.join();
return cash.add(equity).add(fund);
}
}
Sequential execution: Cash: 200ms Equity: 300ms Fund: 150ms Total: 650ms Parallel execution (scatter-gather): Cash: 200ms ─┐ Equity: 300ms ─┤ all run concurrently Fund: 150ms ─┘ Total: 300ms (slowest child) Speedup: 2.2x
How does the fork phase work?
Each child is submitted to a thread pool with supplyAsync, which returns immediately with a CompletableFuture. The actual work runs on an executor thread. After creating three futures, all three are running in parallel.
package com.bank.portfolio;
import java.math.BigDecimal;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutorService;
public class ForkPhaseExample {
public void demonstrate(String customerId,
ExecutorService executor) {
// Each supplyAsync returns IMMEDIATELY
// The lambda runs on an executor thread
CompletableFuture<BigDecimal> f1 =
CompletableFuture.supplyAsync(
() -> fetchBalance(customerId), executor);
CompletableFuture<BigDecimal> f2 =
CompletableFuture.supplyAsync(
() -> fetchEquity(customerId), executor);
// At this point, BOTH are running concurrently
// on executor threads
System.out.println("Both tasks submitted, running in parallel");
}
private BigDecimal fetchBalance(String id) {
// Simulates I/O call to account service
return new BigDecimal("150000.00");
}
private BigDecimal fetchEquity(String id) {
// Simulates I/O call to equity service
return new BigDecimal("250000.00");
}
}
How does the join phase work?
The parent waits for every child with allOf, which returns a future that completes only when all children complete. Always bound with a timeout so one slow dependency cannot hang the request.
package com.bank.portfolio;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
public class JoinPhaseExample {
public void demonstrate(CompletableFuture<?>... children)
throws Exception {
try {
// allOf completes when ALL children complete
// get() with timeout bounds the wait
CompletableFuture.allOf(children)
.get(2, TimeUnit.SECONDS);
System.out.println("All children completed within 2s");
} catch (TimeoutException e) {
System.out.println("At least one child timed out!");
// Cancel remaining children
for (CompletableFuture<?> child : children) {
child.cancel(true);
}
}
}
}
Join behavior:
Child 1: completes in 200ms
Child 2: completes in 300ms
Child 3: completes in 150ms
allOf: completes at 300ms (slowest)
With timeout:
Child 1: completes in 200ms
Child 2: hangs (service down)
Timeout at 2s -> TimeoutException
Action: use fallback for child 2
How does error handling work?
If any child fails, allOf completes exceptionally. Attach exceptionally to each child for per-child fallbacks so one failure does not crash the whole aggregate.
package com.bank.portfolio;
import java.math.BigDecimal;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutorService;
public class ErrorHandlingExample {
public BigDecimal getNetWorthGraceful(String customerId,
ExecutorService executor) {
CompletableFuture<BigDecimal> cashF =
CompletableFuture.supplyAsync(
() -> accountService.getTotalBalance(customerId),
executor);
// No fallback: cash is critical, fail the whole thing
CompletableFuture<BigDecimal> equityF =
CompletableFuture.supplyAsync(
() -> equityService.getHoldingsValue(customerId),
executor)
.exceptionally(ex -> {
// Fallback: equity service down -> show 0
System.out.println("Equity service down: " +
ex.getMessage());
return BigDecimal.ZERO;
});
CompletableFuture<BigDecimal> fundF =
CompletableFuture.supplyAsync(
() -> fundService.getHoldingsValue(customerId),
executor)
.exceptionally(ex -> BigDecimal.ZERO);
return cashF.thenCombine(equityF, BigDecimal::add)
.thenCombine(fundF, BigDecimal::add)
.join();
}
}
Error handling strategies:
Cash (critical): no fallback -> fail the whole dashboard
Equity (optional): fallback to 0 -> show partial net worth
Fund (optional): fallback to 0 -> show partial net worth
Equity service down:
Net worth = 150000 (cash) + 0 (equity fallback) + 75000 (fund)
Dashboard shows: "Net Worth: 225000 (equity data unavailable)"
Why use a dedicated executor?
The common ForkJoinPool is sized for CPU-bound work. Blocking I/O calls on it can starve the entire application. Use a dedicated bounded executor as a bulkhead.
package com.bank.config;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
@Configuration
public class ExecutorConfig {
@Bean(name = "portfolioExecutor")
public ExecutorService portfolioExecutor() {
// Dedicated pool for portfolio I/O calls
// Does not starve the shared ForkJoinPool
return Executors.newFixedThreadPool(10);
}
}
WRONG: use ForkJoinPool.commonPool()
Shared by all CompletableFuture.supplyAsync() without executor
I/O calls block these threads -> stream().parallel() starves
Other features starve
CORRECT: dedicated bounded executor
Portfolio I/O: 10 threads (bulkhead)
Other features: unaffected
How does thenCombine provide a cleaner composition?
Instead of allOf + join() on each, thenCombine chains the combination functionally.
package com.bank.portfolio;
import java.math.BigDecimal;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutorService;
public class FunctionalComposition {
public CompletableFuture<BigDecimal> getNetWorthAsync(
String customerId, ExecutorService executor) {
CompletableFuture<BigDecimal> cashF =
CompletableFuture.supplyAsync(
() -> accountService.getTotalBalance(customerId),
executor);
CompletableFuture<BigDecimal> equityF =
CompletableFuture.supplyAsync(
() -> equityService.getHoldingsValue(customerId),
executor)
.exceptionally(ex -> BigDecimal.ZERO);
CompletableFuture<BigDecimal> fundF =
CompletableFuture.supplyAsync(
() -> fundService.getHoldingsValue(customerId),
executor)
.exceptionally(ex -> BigDecimal.ZERO);
// Functional composition: no manual join needed
return cashF
.thenCombine(equityF, BigDecimal::add)
.thenCombine(fundF, BigDecimal::add);
}
}
What are common mistakes?
The most common mistake is using the shared ForkJoinPool for blocking I/O. Another is not setting a timeout on the join, which lets one slow child hang the entire request.
WRONG: no timeout
Child 3 hangs forever -> dashboard never loads
Rajesh stares at spinner indefinitely
CORRECT: 2-second timeout
Child 3 hangs -> timeout after 2s -> fallback returns 0
Dashboard loads with partial data in 2s
What should a senior engineer know?
A senior engineer propagates correlation IDs into child tasks for distributed tracing, sizes the dedicated executor based on the number of downstream services and their expected latency, and distinguishes between critical children (whose failure fails the aggregate) and optional children (whose failure degrades gracefully). In reactive stacks, the same pattern uses Mono.zip instead of CompletableFuture.allOf.
package com.bank.portfolio;
public class SeniorGuidelines {
public static void main(String[] args) {
System.out.println("Senior guidelines for scatter-gather:");
System.out.println(" 1. Propagate traceId and paymentId to child threads");
System.out.println(" 2. Size executor: (child count * expected concurrency)");
System.out.println(" 3. Classify children: critical (fail whole) vs optional (fallback)");
System.out.println(" 4. Always set a timeout on the join (allOf.get(2, SECONDS))");
System.out.println(" 5. Never use ForkJoinPool.commonPool() for blocking I/O");
System.out.println(" 6. In reactive: Mono.zip() replaces allOf for non-blocking");
System.out.println(" 7. Monitor: per-child latency, fallback rate, timeout rate");
}
}
Expected output:
Senior guidelines for scatter-gather:
1. Propagate traceId and paymentId to child threads
2. Size executor: (child count * expected concurrency)
3. Classify children: critical (fail whole) vs optional (fallback)
4. Always set a timeout on the join (allOf.get(2, SECONDS))
5. Never use ForkJoinPool.commonPool() for blocking I/O
6. In reactive: Mono.zip() replaces allOf for non-blocking
7. Monitor: per-child latency, fallback rate, timeout rate
Visual Overview
Parent Task: Get Rajesh's Net Worth
│
├── SPLIT into independent children
│
┌────┴────┬──────────┬──────────┐
│ │ │ │
▼ ▼ ▼ │
┌──────┐ ┌───────┐ ┌───────┐ │
│Cash │ │Equity │ │Fund │ │ FORK (supplyAsync)
│200ms │ │300ms │ │150ms │ │ all run concurrently
└──┬───┘ └──┬────┘ └──┬────┘ │
│ │ │ │
└────────┼─────────┘ │
│ │
allOf().get(2s) │ JOIN (wait for all)
│ │
▼ │
cash + equity + fund │ COMBINE
= 475000.00 │
│ │
▼ │
Return net worth │
Total time: 300ms (slowest child, not 650ms sum)
Key Takeaway
A parent task that spawns multiple child tasks and combines their results follows the fork-join or scatter-gather pattern with four phases: split, fork, join, and combine. The parent divides the work into genuinely independent units, submits each to a dedicated thread pool with supplyAsync, waits for all with allOf and a timeout, and merges the results. The key property is that total latency equals the slowest child, not the sum.
At National Bank of India, Rajesh's net-worth dashboard needed his cash balance, equity holdings, and mutual-fund value. The original code fetched them sequentially in 650 milliseconds. With scatter-gather on a dedicated executor, the dashboard loads in 300 milliseconds, the time of the slowest single source. The exceptionally fallback on the equity child means that when the equity-pricing service is down, that source contributes zero and the rest of the net worth still displays.
The senior details are error handling and executor choice. Attach per-child fallbacks with exceptionally so one failure degrades gracefully. Use a dedicated bounded executor for I/O calls, not the shared ForkJoinPool, which is a bulkhead that prevents one feature from starving the application. Always bound the join with a timeout and propagate correlation IDs for tracing.
Remember one sentence: fork independent children with supplyAsync, join them with allOf under a timeout, and combine their results, so latency is the slowest child rather than the sum, with per-child fallbacks making it resilient.
---