Threads rising, CPU low: what is wrong?

• Symptom – thread count is rising steadily, CPU utilization is low (under 10%). Threads are not computing; they are WAITING or TIMED_WAITING.
• Cause 1: slow IO – HTTP or DB calls block for minutes because no timeout is configured. TCP default timeout is 300 seconds. 100 requests/second for 300 seconds = 30,000 threads.
• Cause 2: thread leak – new Thread() per request or Executors.newCachedThreadPool() creates threads without bounds or cleanup.
• Cause 3: connection pool exhaustion – HikariCP maxPoolSize = 10, each query takes 2 seconds. Thread 11 waits in getConnection() indefinitely.
• Wrong fix – "add more threads" or "add more servers" creates more threads waiting on the same slow dependency.
• Right fix – timeout on every IO call, bounded ThreadPoolExecutor, circuit breaker for failing services, fix the slow dependency.

What does rising threads + low CPU indicate?

When thread count increases but CPU stays low, threads are not doing useful work. They are blocked on external IO, waiting for resources, or leaked and never terminated. The threads consume memory (each platform thread reserves about 1 MB of stack) but produce no output. This symptom is a clear signal that the problem is IO, not compute.

How do you investigate with jstack?

jstack <PID> > dump.txt
# Count thread states:
grep -c 'TIMED_WAITING' dump.txt    # threads sleeping or on timeout
grep -c 'WAITING' dump.txt           # threads waiting for signals
grep 'SocketInputStream.read' dump.txt  # slow HTTP calls
grep 'HikariPool.getConnection' dump.txt  # DB pool exhausted

What are the three root causes?

Cause 1: Slow IO without timeout. Each request thread blocks on a slow HTTP or DB call with no configured timeout. The default TCP timeout can be 300 seconds. Over 5 minutes at 100 requests/second, that creates 30,000 blocked threads, each consuming 1 MB of stack memory.

Cause 2: Thread leak. Creating a new Thread() per request or using newCachedThreadPool() without controlling the task duration. CachedThreadPool creates a new thread whenever no idle thread is available. If tasks hang, threads accumulate without limit.

Cause 3: Connection pool exhaustion. All database connections are in use by slow queries. New threads wait in HikariPool.getConnection() indefinitely, consuming stack memory but doing no work. With a maxPoolSize of 10 and each query taking 2 seconds, thread 11 waits until a connection becomes available.

Why is "add more threads/servers" the wrong fix?

More threads create more connections to the same slow dependency, making it even slower (more load on an overloaded service). More servers multiply the problem by creating more threads that all wait on the same bottleneck. The root cause is the slow dependency, not insufficient capacity.

What is the correct fix?

Three changes together solve the problem:

package com.bank.service;

import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.net.URI;
import java.time.Duration;

public class TimeoutFixDemo {
    // 1. Always configure HTTP client timeouts
    private static final HttpClient client = HttpClient.newBuilder()
            .connectTimeout(Duration.ofSeconds(5))
            .build();

    public String callExternalService(String url) throws Exception {
        HttpRequest request = HttpRequest.newBuilder()
            .uri(URI.create(url))
            .timeout(Duration.ofSeconds(5))  // per-request timeout
            .build();
        return client.send(request, HttpResponse.BodyHandlers.ofString()).body();
    }
}

package com.bank.pool;

import java.util.concurrent.*;

public class BoundedPoolFix {
    // 2. Use bounded thread pool instead of CachedThreadPool
    private static final ExecutorService pool = new ThreadPoolExecutor(
        10, 20, 60L, TimeUnit.SECONDS,
        new ArrayBlockingQueue<>(500),
        new ThreadPoolExecutor.CallerRunsPolicy()
    );
    // CallerRunsPolicy provides natural backpressure
}

// 3. Add circuit breaker for failing services
// When the downstream service is unhealthy,
// fail fast instead of blocking threads
circuitBreaker.executeSupplier(() -> callPaymentGateway(paymentId));

How do you find thread leaks?

Compare thread dumps taken 30 seconds apart. If thread count only grows and never shrinks, threads are being created but never terminated. Look for thread names that do not follow a pool naming pattern, which indicates new Thread() usage.

What is a circuit breaker?

A circuit breaker monitors the success and failure rate of calls to a downstream service. When the failure rate exceeds a threshold (e.g., 50%), the circuit opens and all subsequent calls fail immediately (fast-fail in milliseconds) without attempting the request. After a cooldown period, the circuit enters a half-open state and allows a few probe calls. If probes succeed, the circuit closes.

What are common interview follow-up questions?

• How do you set timeouts for JDBC connections? Configure HikariCP's connectionTimeout (wait for connection from pool) and socketTimeout (wait for DB response). Both should be set to 5-10 seconds.
• How do virtual threads change this problem? Virtual threads do not reserve 1 MB of stack each, so 30,000 virtual threads use far less memory. But the underlying problem (slow dependency) still exists and still needs timeouts and circuit breakers.
• What monitoring should you set up? Alert on rising thread count, alert when pool queue depth exceeds a threshold, and alert when downstream P99 latency exceeds the timeout value.