What is concurrency vs parallelism?

• Concurrency – multiple tasks make progress by interleaving their execution; the OS switches between threads rapidly (time-slicing), giving the illusion of simultaneous execution; possible on a single core.
• Parallelism – multiple tasks execute at the exact same instant on different CPU cores; requires multi-core hardware.
• IO-bound – tasks that spend most time waiting for external resources (DB, network, file system); benefit from concurrency because idle threads yield the CPU.
• CPU-bound – tasks that spend most time computing (encryption, sorting, compression); benefit from parallelism because they need all cores busy.
• Thread count rule – IO-bound: cores x (1 + waitTime/computeTime); CPU-bound: cores (or cores + 1).

What is concurrency?

Concurrency means multiple tasks are making progress within overlapping time periods. On a single-core CPU, the operating system uses time-slicing to switch between threads rapidly, giving the appearance of simultaneous execution. Concurrency is about structure: designing your program to handle many things at once.

What is parallelism?

Parallelism means multiple tasks are executing at the exact same instant on different CPU cores. It requires multi-core hardware. Parallelism is about execution: actually doing many things at once.

How does IO-bound work benefit from concurrency?

IO-bound tasks spend most of their time waiting for external resources: database queries, HTTP calls, file reads. While one thread waits for a response, other threads can use the CPU. Adding more threads than cores improves throughput because most threads are idle most of the time.

package com.bank.service;

import java.util.concurrent.*;
import java.util.*;

public class IoBoundDemo {
    public static void main(String[] args) throws Exception {
        // IO-bound: 100 HTTP calls, each 200ms wait
        // Sequential: 100 x 200ms = 20 seconds
        // Concurrent: all 100 calls overlap, ~200ms total

        ExecutorService ioPool = Executors.newFixedThreadPool(100);
        List<Future<String>> futures = new ArrayList<>();

        long start = System.currentTimeMillis();
        for (int i = 0; i < 100; i++) {
            final int id = i;
            futures.add(ioPool.submit(() -> {
                Thread.sleep(200); // simulate HTTP call
                return "Balance for SAV" + id;
            }));
        }

        for (Future<String> f : futures) {
            f.get(); // all complete in ~200ms
        }
        long elapsed = System.currentTimeMillis() - start;
        System.out.println("100 IO calls in " + elapsed + "ms");

        ioPool.shutdown();
    }
}

100 IO calls in 210ms

How does CPU-bound work benefit from parallelism?

CPU-bound tasks spend most of their time computing. Adding more threads than cores adds context-switching overhead without improving throughput. The optimal thread count equals the number of available cores.

package com.bank.service;

import java.util.stream.LongStream;

public class CpuBoundDemo {
    public static void main(String[] args) {
        int cores = Runtime.getRuntime().availableProcessors();
        System.out.println("Available cores: " + cores);

        // CPU-bound: parallel computation on all cores
        long start = System.currentTimeMillis();
        long sum = LongStream.range(0, 10_000_000)
            .parallel() // uses ForkJoinPool with cores threads
            .map(x -> x * x % 17) // CPU-bound work
            .sum();
        long elapsed = System.currentTimeMillis() - start;
        System.out.println("Parallel sum in " + elapsed + "ms: " + sum);
    }
}

How do you calculate the optimal thread count?

The formula depends on the workload type:

• IO-bound: threads = cores x (1 + waitTime / computeTime)
• 4 cores, 200ms wait, 10ms compute: 4 x (1 + 200/10) = 84 threads
• CPU-bound: threads = cores (or cores + 1 for scheduling margin)
• 4 cores: 4 threads

Why should you use separate pools for IO and CPU work?

Mixing IO-bound and CPU-bound tasks in one pool leads to starvation. IO tasks block threads while waiting, leaving no threads for CPU work.

// WRONG: one pool for everything
ExecutorService shared = Executors.newFixedThreadPool(4);
// IO tasks occupy all 4 threads waiting for DB
// CPU tasks queued behind them, starved

// CORRECT: separate pools
ExecutorService ioPool = Executors.newFixedThreadPool(50);  // many threads
ExecutorService cpuPool = Executors.newFixedThreadPool(
    Runtime.getRuntime().availableProcessors());             // cores threads

// IO work on ioPool, CPU work on cpuPool
// Neither starves the other

How is concurrency different from parallelism?

| Aspect | Concurrency | Parallelism |
|---|---|---|
| Definition | Dealing with many things at once | Doing many things at once |
| Cores needed | One is enough | Multiple required |
| Focus | Structure | Execution |
| IO-bound | Ideal (threads yield CPU) | Not needed (waiting, not computing) |
| CPU-bound | Adds overhead (context switching) | Ideal (all cores busy) |
| Thread count | Many (cores x N) | Few (= cores) |

What about virtual threads (Java 21)?

Virtual threads are lightweight threads managed by the JVM. You can create millions of them. When a virtual thread blocks on IO, the JVM unmounts it from the carrier (platform) thread, freeing the carrier for other work. This makes simple blocking code scale like async code for IO-bound workloads.

// Java 21: one virtual thread per task
try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
    for (int i = 0; i < 100_000; i++) {
        executor.submit(() -> {
            Thread.sleep(200); // blocking IO, but cheap with virtual threads
        });
    }
}

What are common interview follow-up questions?

• Can you have concurrency without parallelism? Yes, on a single-core CPU with time-slicing.
• Can you have parallelism without concurrency? Yes, SIMD instructions process multiple data items in lockstep.
• What determines whether a task is IO-bound or CPU-bound? Measure the ratio of wait time to compute time.