Java Thread Pools

Creating a thread is expensive. Each new Thread() allocates about 1 MB of native stack, asks the OS to schedule a new kernel thread, and adds load to GC. A program that hands out one thread per task is fine for ten tasks; it falls over at ten thousand. The fix is a thread pool — a small set of long-lived worker threads that pull tasks from a queue. The pool owns the threads; you own the tasks.

This chapter is the conceptual one — what a pool is, the knobs that configure it, and the failure modes. The next chapter, Executor framework, introduces the Executor/ExecutorService types you use to talk to a pool. The two are intertwined; this chapter focuses on the what and why, the next on the how.

Why pool?

Three problems a pool solves:

1. Thread-creation cost. Allocating a native stack and asking the OS for a new thread is on the order of milliseconds. Reusing existing threads is microseconds. At scale, the difference is the difference between a server that holds up under load and one that doesn't.
2. Resource limits. A platform thread on a 64-bit JVM takes about 1 MB of stack — 64 GB of RAM is ~64,000 threads, and the OS has its own per-thread overhead. Unbounded thread creation is unbounded RAM consumption. A pool puts a cap on the count.
3. Predictable parallelism. A pool with N workers gives you exactly N parallel tasks. That's a much better fit for "use all 16 cores" than "create one thread per request and hope."

The cost of pooling: you have to size it. Too small → tasks queue up and latency grows. Too big → context switching dominates and throughput drops. The sizing chapter (executor framework) covers the rules of thumb; this chapter is about what the pool is.

The anatomy of a pool

A thread pool is essentially three things:

1. A bounded set of worker threads. Workers run a loop: take a task from the queue, run it, take the next one, repeat. They live for the lifetime of the pool (or until idle for too long, depending on policy).
2. A task queue. When you submit work and no worker is free, the task goes here. The queue type — LinkedBlockingQueue, ArrayBlockingQueue, SynchronousQueue — affects how the pool grows under load.
3. A submission API. execute(Runnable), submit(Callable), invokeAll(...) — the ways you put work in the pool.

In Java, all of that is wrapped in java.util.concurrent.ThreadPoolExecutor, which is the underlying class for nearly every pool you'll meet.

The seven knobs of ThreadPoolExecutor

Direct construction (which you rarely do, but the parameters are what every factory passes underneath):

new ThreadPoolExecutor(
  int corePoolSize,
  int maximumPoolSize,
  long keepAliveTime, TimeUnit unit,
  BlockingQueue<Runnable> workQueue,
  ThreadFactory threadFactory,
  RejectedExecutionHandler handler
);

The non-obvious interaction: the pool prefers to fill the queue before creating new threads above core. So an unbounded queue means the pool never grows past core — it just queues up forever. A bounded queue (or SynchronousQueue) is what makes the max setting meaningful.

The four rejection policies

When submit can't accept a task (queue full, max workers all busy), the RejectedExecutionHandler decides what happens:

Default-throwing is usually the safe choice. CallerRunsPolicy is a clever back-pressure mechanism — when the pool is overwhelmed, the submitter is slowed down to match, which naturally rate-limits the source.

The Executors factory methods — and why you should mostly avoid them

java.util.concurrent.Executors ships convenience factories:

Executors.newFixedThreadPool(n);             // core = max = n, unbounded LinkedBlockingQueue
Executors.newCachedThreadPool();             // core = 0, max = Integer.MAX_VALUE, SynchronousQueue, 60s keep-alive
Executors.newSingleThreadExecutor();         // fixed pool with one thread
Executors.newScheduledThreadPool(n);         // for delay/repeat scheduling
Executors.newVirtualThreadPerTaskExecutor(); // Java 21+: one virtual thread per task

Two of these have well-known traps:

• newFixedThreadPool uses an unbounded LinkedBlockingQueue. Under sustained overload, the queue grows without limit — eventually OOM. The pool size is fixed; the work piling up behind it is not.
• newCachedThreadPool has maximum = Integer.MAX_VALUE. Under a sustained burst of work, it creates threads without limit — eventually exhausts the OS's per-process thread limit and crashes the JVM.

These are fine for small jobs, demos, and one-off scripts. For production code, build a ThreadPoolExecutor directly with a bounded queue, a sensible max, and an explicit rejection policy.

The exception: newVirtualThreadPerTaskExecutor (Java 21+) hands out virtual threads, which are cheap enough that "one per task" actually works. We cover this in the virtual threads chapter.

Lifecycle: shutdown vs shutdownNow

A pool keeps running until you tell it to stop. The two stop modes:

pool.shutdown();                              // stop accepting new work; let queued tasks finish
pool.shutdownNow();                           // stop accepting; interrupt running threads; return queued tasks

boolean terminated = pool.awaitTermination(10, TimeUnit.SECONDS);

shutdown is the polite version: no new submissions accepted, existing work finishes, then the pool exits. shutdownNow is the rude version: interrupt the workers, return the pending queue. Use shutdown for clean exit; use shutdownNow after a shutdown + awaitTermination deadline if work didn't finish.

The combined shutdown pattern from the JDK docs:

pool.shutdown();
try {
  if (!pool.awaitTermination(10, TimeUnit.SECONDS)) {
    pool.shutdownNow();
    pool.awaitTermination(5, TimeUnit.SECONDS);
  }
} catch (InterruptedException e) {
  pool.shutdownNow();
  Thread.currentThread().interrupt();
}

You almost always want this exact shape in any code that owns a pool. Without shutdown, the JVM keeps the workers alive (non-daemon by default) and doesn't exit.

Naming workers via ThreadFactory

The default Executors.defaultThreadFactory() names threads pool-1-thread-1, pool-1-thread-2, etc. That's a tiny step up from Thread-7 but still not great. Production code uses a named factory:

ThreadFactory factory = r -> {
  Thread t = new Thread(r, "image-worker-" + COUNTER.incrementAndGet());
  t.setDaemon(false);
  t.setUncaughtExceptionHandler((thr, ex) -> log.error("uncaught in " + thr.getName(), ex));
  return t;
};
ExecutorService pool = new ThreadPoolExecutor(
    4, 4, 0, TimeUnit.SECONDS,
    new LinkedBlockingQueue<>(1000),
    factory,
    new ThreadPoolExecutor.CallerRunsPolicy());

The factory is your chance to set every per-thread property: name, daemon flag, priority, uncaught-exception handler, thread-group. In a 200-thread heap dump, a thread called image-worker-7 is a thread you can find.

A worked example: build a bounded pool with back-pressure

The program below builds a ThreadPoolExecutor with 4 workers, a bounded queue of 8, and the CallerRunsPolicy rejection handler — so the submitter is slowed down when the pool is overwhelmed instead of throwing.

What to take from the run:

• The pool had a strict cap of 4 worker threads. With 40 tasks at 50 ms each, the idealised serial-by-pool time is 40 * 50 / 4 = 500 ms. The actual wall-clock was close to that — minus the cost of the CallerRunsPolicy slowing the submitter down whenever the queue filled.
• Some tasks reported a thread name of main. That's CallerRunsPolicy in action: when the queue was full and all workers were busy, pool.execute ran the task on the calling thread instead of queueing or throwing. The submitter got slower; the system stayed bounded. That's back-pressure done right.
• pool.getLargestPoolSize() was 4 — the maximum stayed equal to the core. The pool didn't grow past core even under sustained load because the bounded queue had room for the brief bursts. With an unbounded queue (the Executors.newFixedThreadPool default), the queue would have accepted every task and largestPoolSize would have stayed at 4 — but memory would have spiked while tasks accumulated.
• The shutdown sequence is the production pattern. shutdown() told the pool to stop accepting new submissions; awaitTermination(5, SECONDS) waited up to 5 seconds for in-flight work; if work hadn't finished, shutdownNow() would have interrupted the remaining workers. Without these calls, the JVM doesn't exit — the non-daemon workers keep it alive.
• The thread factory gave every worker a meaningful name (worker-1 ... worker-4) and an uncaught-exception handler. In a production thread dump or profiler, those names are the difference between "I know which subsystem this is" and "I have no idea." Set them on every pool you create.

What's next

The next chapter, Java Executor Framework, introduces the type hierarchy you use to talk to thread pools — Executor, ExecutorService, ScheduledExecutorService — and how to size a pool for CPU-bound and I/O-bound workloads.

Practice