Definition
- Thread pool – a fixed set of worker threads that process tasks from a shared queue. Workers are reused across tasks, avoiding the cost of creating and destroying a thread per request.
- ExecutorService – the interface for managing a thread pool. Key methods:
submit(Runnable/Callable),shutdown(),shutdownNow(),awaitTermination(). - Executors factory – convenience methods:
newFixedThreadPool(n),newCachedThreadPool(),newSingleThreadExecutor(),newScheduledThreadPool(n). - ThreadPoolExecutor – the underlying implementation. In production, use it directly for full control over core/max pool size, queue bounds, keep-alive time, and rejection policy.
Creating a new thread per request is like hiring a new teller for every customer and firing them after one transaction. 10,000 customers means 10,000 hires and fires. A thread pool is hiring 10 permanent tellers at branch opening. Customers queue up. Each teller serves one customer, then takes the next from the queue. Fixed pool: exactly 10 tellers, always. Cached pool: tellers hired on demand, dismissed after 60 seconds of idleness. Scheduled pool: tellers who handle appointments at specific times.
Understanding the Concepts
Why use a thread pool?
Creating raw threads with new Thread().start() is dangerous at scale.
package com.bank.service;
public class WhyThreadPool {
// WRONG: unbounded thread creation
public static void handleRequest(Runnable task) {
new Thread(task).start();
// 10K requests -> 10K threads -> 10GB stack -> OutOfMemoryError!
// No limit, no reuse, no monitoring, no graceful shutdown.
}
// CORRECT: bounded thread pool
private static final java.util.concurrent.ExecutorService pool =
java.util.concurrent.Executors.newFixedThreadPool(100);
public static void handleRequestSafely(Runnable task) {
pool.submit(task);
// 10K requests queued, 100 threads process them, reused.
}
}
What are the Executors factory types?
package com.bank.service;
import java.util.concurrent.*;
public class PoolTypesDemo {
public static void main(String[] args) {
// Fixed: n workers, unbounded LinkedBlockingQueue
ExecutorService fixed = Executors.newFixedThreadPool(10);
fixed.submit(() -> System.out.println("Fixed pool"));
// Cached: 0 to Integer.MAX_VALUE threads, SynchronousQueue
ExecutorService cached = Executors.newCachedThreadPool();
cached.submit(() -> System.out.println("Cached pool"));
// Single: 1 worker, tasks execute sequentially
ExecutorService single = Executors.newSingleThreadExecutor();
single.submit(() -> System.out.println("Single pool"));
// Scheduled: n workers for delayed/periodic tasks
ScheduledExecutorService scheduled = Executors.newScheduledThreadPool(5);
scheduled.schedule(() -> System.out.println("Delayed"),
5, TimeUnit.SECONDS);
scheduled.scheduleAtFixedRate(() -> System.out.println("Health check"),
0, 30, TimeUnit.SECONDS);
fixed.shutdown();
cached.shutdown();
single.shutdown();
}
}
Why are Executors factory methods dangerous in production?
| Factory method | Danger |
|---|---|
| newFixedThreadPool(n) | Unbounded LinkedBlockingQueue: OOM if tasks pile up faster than workers process them |
| newCachedThreadPool() | Unbounded thread creation: up to Integer.MAX_VALUE threads |
| newSingleThreadExecutor() | Unbounded queue: same OOM risk |
How should you create a thread pool in production?
Use ThreadPoolExecutor directly with bounded queue and rejection policy.
package com.bank.service;
import java.util.concurrent.*;
public class ProductionThreadPool {
private static final ThreadPoolExecutor pool = new ThreadPoolExecutor(
10, // corePoolSize
50, // maxPoolSize (scale up under load)
60, TimeUnit.SECONDS, // idle threads above core die after 60s
new LinkedBlockingQueue<>(1000), // bounded queue: max 1000 pending tasks
new ThreadPoolExecutor.CallerRunsPolicy() // rejection: caller thread executes
);
public static void submitTask(Runnable task) {
pool.submit(task);
}
}
The CallerRunsPolicy provides natural back-pressure: when the queue is full and all threads are busy, the submitting thread executes the task itself, slowing down the producer.
How do you size a thread pool?
- CPU-bound tasks (computation, sorting, encryption): pool size equals the number of CPU cores. More threads cause context switching overhead without benefit.
- IO-bound tasks (database calls, HTTP requests, file IO): pool size equals cores multiplied by 2 to 4, because threads spend most of their time waiting for IO.
- Formula:
poolSize = cores * (1 + waitTime / computeTime)
package com.bank.service;
public class PoolSizing {
// CPU-bound: use core count
static int cpuBoundPoolSize() {
return Runtime.getRuntime().availableProcessors();
}
// IO-bound: use cores * 2-4
static int ioBoundPoolSize() {
int cores = Runtime.getRuntime().availableProcessors();
return cores * 3; // 3x for typical DB-heavy banking workload
}
}
How do you shut down a thread pool gracefully?
package com.bank.service;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.TimeUnit;
public class GracefulShutdown {
public static void shutdown(ExecutorService pool) {
pool.shutdown(); // stop accepting new tasks
try {
if (!pool.awaitTermination(30, TimeUnit.SECONDS)) {
pool.shutdownNow(); // force-stop running tasks
if (!pool.awaitTermination(10, TimeUnit.SECONDS)) {
System.err.println("Pool did not terminate");
}
}
} catch (InterruptedException e) {
pool.shutdownNow();
Thread.currentThread().interrupt();
}
}
}
What about virtual threads (Java 21)?
For IO-bound workloads, virtual threads eliminate the need for thread pool sizing. You can create millions of virtual threads because they are managed by the JVM, not the OS.
package com.bank.service;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
public class VirtualThreadPool {
public static void main(String[] args) {
try (ExecutorService pool = Executors.newVirtualThreadPerTaskExecutor()) {
for (int i = 0; i < 100000; i++) {
pool.submit(() -> {
// Each virtual thread is lightweight: ~1KB vs 1MB for platform threads
System.out.println("Trade on " + Thread.currentThread());
});
}
}
}
}
What should a senior engineer know?
- Rejection policies:
AbortPolicy(throws RejectedExecutionException),CallerRunsPolicy(caller executes),DiscardPolicy(silently drops),DiscardOldestPolicy(drops oldest queued task). - Thread factory: use a custom
ThreadFactoryto name threads (e.g., "settlement-worker-1") and set daemon status. Named threads make thread dumps readable. - Monitoring: expose pool metrics via JMX or Micrometer:
getActiveCount(),getQueue().size(),getCompletedTaskCount(). Alert when queue size exceeds a threshold. - ForkJoinPool: specialized for recursive divide-and-conquer tasks. Work-stealing: idle threads steal tasks from busy threads' deques. Used by parallel streams and CompletableFuture.
Visual Overview
ThreadPoolExecutor Internals:
submit(task) ──▶ ┌────────────────────┐
│ Task Queue (bounded) │
│ [task1][task2][...] │
└──────────┬───────────┘
│
┌─────────▼──────────┐
│ Worker Threads │
│ ┌──────┐ ┌──────┐ │
│ │ W-1 │ │ W-2 │ │ core threads
│ └──────┘ └──────┘ │
│ ┌──────┐ ┌──────┐ │
│ │ W-3 │ │ W-4 │ │ max threads (created under load)
│ └──────┘ └──────┘ │
└─────────────────────┘
Queue full + all threads busy:
┌──────────────────────────────────┐
│ Rejection Policy │
│ CallerRunsPolicy: caller does it │
│ AbortPolicy: throw exception │
│ DiscardPolicy: drop silently │
└──────────────────────────────────┘
Key Takeaway
A thread pool pre-creates reusable worker threads managed by an ExecutorService. Benefits: thread reuse (creation cost paid once), bounded concurrency (no OOM from unlimited threads), task queueing (excess work waits), and lifecycle management (graceful shutdown).
In production, never use Executors factory methods directly because they create unbounded queues or unbounded thread counts. Use ThreadPoolExecutor directly with a bounded queue and a rejection policy. Size the pool based on workload type: CPU-bound tasks use core count threads, IO-bound tasks use cores multiplied by 2 to 4.
Java 21 virtual threads change the game for IO-bound workloads: you can create millions of virtual threads without pool sizing concerns, because each virtual thread consumes roughly 1KB instead of 1MB.
Remember one sentence: use ThreadPoolExecutor with a bounded queue and rejection policy in production, and never create raw threads with new Thread().start().
---