Definition
- Thread-per-request model -- each incoming HTTP request gets one exclusive thread from Tomcat's thread pool for its entire lifecycle, from request parsing to response writing.
- Tomcat thread pool -- default 200 threads (
server.tomcat.threads.max); configurable inapplication.yml. - Singleton thread safety -- controllers, services, and repositories are singletons shared by all threads; mutable instance fields create race conditions.
- Local variable safety -- variables declared inside a method live on the thread's stack and are inherently thread-safe.
- Thread pool exhaustion -- when slow downstream calls hold threads without timeouts, the pool fills up and new requests are rejected even though CPU is idle.
National Bank of India has 200 teller windows (thread pool). Each customer (HTTP request) gets one window for their entire visit. All tellers share ONE calculator (singleton bean). If teller Rajesh writes "50000" on the calculator and teller Priya writes "75000" at the same time, the result is corrupted (race condition). Each teller's personal notepad (local variables) is safe. If one teller makes a phone call that takes 30 minutes (slow downstream call), that window is blocked for other customers.
Understanding the Concepts
What is the thread-per-request model?
Spring MVC runs on a Servlet container, Tomcat by default. Tomcat maintains a thread pool (default 200 threads). When an HTTP request arrives, Tomcat assigns one thread from the pool to handle it. That thread processes the entire request: parsing headers, deserializing the body, executing the controller method, calling services, querying the database, serializing the response, and writing it back to the client. After the response is sent, the thread returns to the pool.
# application.yml -- configuring Tomcat thread pool
server:
tomcat:
threads:
max: 200 # maximum threads
min-spare: 10 # minimum idle threads
accept-count: 100 # queue when all threads are busy
connection-timeout: 20000 # 20 seconds
One slow request does not block others as long as free threads remain. If 199 threads are handling requests and one is slow, the 200th thread can still serve a new customer. The problem starts when ALL 200 threads are busy.
Why are singleton beans a thread-safety concern?
Controllers, services, and repositories are singletons by default. One instance serves ALL 200 threads concurrently. This means:
- Final injected fields are safe because they are assigned once during construction and never changed.
- Local variables are safe because each thread has its own stack frame.
- Mutable instance fields are DANGEROUS because multiple threads read and write them concurrently.
package com.bank.service;
import org.springframework.stereotype.Service;
import java.math.BigDecimal;
@Service
public class TransferService {
private final AccountRepository repo; // SAFE: final, assigned once
// DANGEROUS: shared mutable state
private String lastTransferFrom; // Thread A writes, Thread B reads
private BigDecimal lastAmount; // Race condition!
public TransferService(AccountRepository repo) {
this.repo = repo;
}
public void transfer(String from, String to, BigDecimal amount) {
this.lastTransferFrom = from; // Thread A: "SAV2001"
// Thread B executes here: this.lastTransferFrom = "SAV2002"
this.lastAmount = amount;
// lastTransferFrom is now "SAV2002" for BOTH threads!
BigDecimal localBalance = repo.getBalance(from); // SAFE: local variable
}
}
What happens when you store per-request state in a singleton?
Storing per-request state in a singleton instance field is a race condition bug. Multiple threads overwrite each other's data.
WRONG -- mutable instance field in a singleton:
package com.bank.controller;
import org.springframework.web.bind.annotation.*;
import java.math.BigDecimal;
@RestController
public class TransferController {
private String currentUser; // shared by 200 threads!
@PostMapping("/transfers")
public String transfer(@RequestParam String userId,
@RequestParam BigDecimal amount) {
this.currentUser = userId; // Thread 1: "Rajesh"
// Context switch: Thread 2 sets currentUser = "Priya"
// Thread 1 resumes: currentUser is now "Priya"!
return "Transfer by " + this.currentUser; // WRONG user!
}
}
// Thread 1 (Rajesh): POST /transfers?userId=Rajesh&amount=10000
// Thread 2 (Priya): POST /transfers?userId=Priya&amount=5000
// Thread 1 response: "Transfer by Priya" <-- WRONG!
// Thread 2 response: "Transfer by Priya" <-- correct by accident
CORRECT -- use local variables or method parameters:
package com.bank.controller;
import org.springframework.web.bind.annotation.*;
import java.math.BigDecimal;
@RestController
public class TransferController {
private final TransferService transferService; // final, safe
public TransferController(TransferService transferService) {
this.transferService = transferService;
}
@PostMapping("/transfers")
public String transfer(@RequestParam String userId,
@RequestParam BigDecimal amount) {
// userId is a local variable on THIS thread's stack -- safe
transferService.transfer(userId, "SAV2002", amount);
return "Transfer by " + userId; // always correct
}
}
How does thread pool exhaustion happen?
When a downstream call has no timeout, the calling thread blocks indefinitely. At sufficient request rates, all threads are blocked waiting for responses, and new requests are rejected.
package com.bank.service;
import org.springframework.stereotype.Service;
import org.springframework.web.client.RestTemplate;
@Service
public class CreditScoreService {
private final RestTemplate restTemplate;
public CreditScoreService(RestTemplate restTemplate) {
this.restTemplate = restTemplate;
}
// WRONG -- no timeout on downstream call
public int getScore(String accountId) {
// If the credit bureau is down, this blocks FOREVER
// At 10 req/s, all 200 threads are blocked in 20 seconds
return restTemplate.getForObject(
"http://credit-bureau.nbi.com/score/" + accountId, Integer.class
);
}
}
// Timeline:
// t=0s: 10 threads blocked (10 req/s, credit bureau unresponsive)
// t=10s: 100 threads blocked
// t=20s: 200 threads blocked -- ALL THREADS EXHAUSTED
// t=21s: New requests rejected: "Connection refused"
// CPU is 0% -- server is IDLE but UNRESPONSIVE
CORRECT -- always set explicit timeouts:
package com.bank.config;
import org.springframework.boot.web.client.RestTemplateBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.web.client.RestTemplate;
import java.time.Duration;
@Configuration
public class RestTemplateConfig {
@Bean
public RestTemplate restTemplate(RestTemplateBuilder builder) {
return builder
.connectTimeout(Duration.ofSeconds(3)) // connect in 3s or fail
.readTimeout(Duration.ofSeconds(5)) // read in 5s or fail
.build();
}
}
What are thread-safe patterns for shared state?
When you genuinely need shared mutable state in a singleton, use thread-safe structures:
package com.bank.service;
import org.springframework.stereotype.Service;
import java.math.BigDecimal;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.atomic.AtomicLong;
@Service
public class TransferCounterService {
// Thread-safe structures for shared state
private final AtomicLong transferCount = new AtomicLong(0);
private final ConcurrentHashMap<String, BigDecimal> balanceCache =
new ConcurrentHashMap<>();
public void recordTransfer(String accountId, BigDecimal amount) {
transferCount.incrementAndGet(); // atomic, no lock needed
balanceCache.put(accountId, amount); // concurrent-safe
}
public long getTransferCount() {
return transferCount.get();
}
}
For per-request data (like a request trace ID or authenticated user), use request-scoped beans or ThreadLocal:
package com.bank.context;
import org.springframework.stereotype.Component;
import org.springframework.web.context.annotation.RequestScope;
@Component
@RequestScope // new instance per HTTP request
public class RequestContext {
private String traceId;
private String userId;
public String getTraceId() { return traceId; }
public void setTraceId(String traceId) { this.traceId = traceId; }
public String getUserId() { return userId; }
public void setUserId(String userId) { this.userId = userId; }
}
What are common mistakes with servlet concurrency?
WRONG -- using a non-thread-safe HashMap in a singleton:
package com.bank.service;
import org.springframework.stereotype.Service;
import java.math.BigDecimal;
import java.util.HashMap;
import java.util.Map;
@Service
public class RateCacheService {
// WRONG: HashMap is NOT thread-safe
private final Map<String, BigDecimal> cache = new HashMap<>();
public void updateRate(String currency, BigDecimal rate) {
cache.put(currency, rate); // ConcurrentModificationException or data loss
}
}
CORRECT -- use ConcurrentHashMap:
package com.bank.service;
import org.springframework.stereotype.Service;
import java.math.BigDecimal;
import java.util.concurrent.ConcurrentHashMap;
import java.util.Map;
@Service
public class RateCacheService {
// CORRECT: ConcurrentHashMap is thread-safe
private final Map<String, BigDecimal> cache = new ConcurrentHashMap<>();
public void updateRate(String currency, BigDecimal rate) {
cache.put(currency, rate); // thread-safe
}
}
What should a senior engineer know?
- Virtual threads (Java 21). Spring Boot 3.2+ supports virtual threads with
spring.threads.virtual.enabled=true. Each request gets a lightweight virtual thread instead of a platform thread, eliminating pool exhaustion concerns for blocking I/O. - WebFlux reactive model. Spring WebFlux uses a non-blocking event loop with a small number of threads (typically CPU cores). It handles thousands of concurrent connections but requires a fully reactive programming model with
MonoandFlux. - Tomcat vs Jetty vs Undertow. All three use the thread-per-request model with similar defaults. The choice rarely affects concurrency behavior.
- ThreadLocal cleanup. If you use
ThreadLocalfor request data, you must clear it in a filter'sfinallyblock. Thread pool threads are reused, so staleThreadLocaldata leaks to the next request. - Observability. Monitor
tomcat.threads.busyandtomcat.threads.config.maxvia Micrometer and Actuator. Set an alert when busy threads exceed 80% of the maximum.
What are common interview follow-up questions?
- How many requests can a Spring Boot application handle concurrently? (Default 200, configurable via
server.tomcat.threads.max.) - What happens when all threads are busy? (New requests queue up to
accept-count, then are rejected with "Connection refused".) - Is
@RequestScopethread-safe? (Yes, each request gets its own instance.) - How do virtual threads change the concurrency model? (Each request gets a cheap virtual thread; pool exhaustion is no longer a concern for blocking I/O.)
- Why is CPU idle during pool exhaustion? (All threads are blocked waiting for I/O, not doing computation.)
Visual Overview
HTTP Request Flow:
Client ──→ Tomcat Thread Pool (200 threads)
│
├── Thread-1 → Controller → Service → Repository → DB
├── Thread-2 → Controller → Service → REST call → timeout!
├── Thread-3 → Controller → Service → Repository → DB
│ ...
└── Thread-200
Singleton Sharing:
┌──────────────────────────────────────────┐
│ TransferService (ONE instance) │
│ final AccountRepository repo; ← SAFE │
│ String lastUser; ← RACE! │
│ │
│ Thread-1 ──→ lastUser = "Rajesh" │
│ Thread-2 ──→ lastUser = "Priya" │
│ Thread-1 reads lastUser → "Priya" ← BUG│
└──────────────────────────────────────────┘
Pool Exhaustion:
┌─────────────────────────────────┐
│ 200/200 threads BLOCKED │
│ (waiting for downstream call) │
│ CPU: 0% Memory: fine │
│ New requests: REJECTED │
└─────────────────────────────────┘
Key Takeaway
Spring MVC uses Tomcat's thread-per-request model with a default pool of 200 threads. Each incoming HTTP request gets an exclusive thread for its entire lifecycle: parsing, processing, and responding. Singleton beans including controllers, services, and repositories are shared across all 200 threads simultaneously, so they must be stateless or use thread-safe structures. Final fields holding injected dependencies are safe because they are assigned once during construction. Mutable instance fields are the danger: writing per-request data to a singleton field creates a race condition where Rajesh's data is overwritten by Priya's concurrent request.
In the National Bank of India's order matching engine, the TransferService is a singleton that handles hundreds of concurrent transfers. All per-request state lives in local variables on each thread's stack. Shared mutable state uses thread-safe structures: ConcurrentHashMap for the exchange rate cache and AtomicLong for the transfer counter. Every downstream HTTP client in the application is configured with explicit connect and read timeouts, because a call without a timeout holds the Tomcat thread indefinitely. At 10 requests per second with an unresponsive downstream service, all 200 threads are blocked in 20 seconds, making the application completely unresponsive even though CPU is at zero percent.
The key rule is: singleton beans must be stateless. Use local variables for per-request data, ConcurrentHashMap or AtomicReference for shared counters and caches, request-scoped beans for per-request context, and always set explicit timeouts on every downstream call.
Remember one sentence: singleton beans are shared by all threads, so never store per-request state in instance fields, and always set timeouts on downstream calls.
---