Java Executor Framework

The previous chapter described what a thread pool is. This chapter is about the type hierarchy you use to talk to one — the Executor, ExecutorService, and ScheduledExecutorService interfaces. Together they're called the executor framework, introduced in Java 5 to decouple "the work" from "the threads that run it." You write Callable<Result> and Runnable; you submit; the framework handles thread allocation, queueing, and result handoff.

The three-level hierarchy

Executor          // execute(Runnable)
   |
ExecutorService   // + submit/invokeAll/invokeAny/shutdown/awaitTermination
   |
ScheduledExecutorService  // + schedule/scheduleAtFixedRate/scheduleWithFixedDelay

You program to the most general interface that has what you need:

• Executor — the one-method base. Use this when you only need to fire and forget. A method parameter typed as Executor is the most general "give me anything that can run a Runnable" contract.
• ExecutorService — the workhorse. Almost all production code uses this type. Adds submit (with a Future result), bulk operations, and lifecycle.
• ScheduledExecutorService — when you need delayed or repeating execution.

Executor.execute — fire and forget

public interface Executor {
  void execute(Runnable command);
}

That's the whole interface. execute takes a Runnable, runs it sometime in the future, returns nothing. If the work throws, you don't find out — the exception goes to the worker thread's uncaught-exception handler.

execute is the right call when:

• The work has no return value.
• You don't need to wait for it or get its result.
• You don't need to cancel it.

For anything richer, use submit.

ExecutorService.submit, the rich version

public interface ExecutorService extends Executor {
  <T> Future<T> submit(Callable<T> task);
  Future<?> submit(Runnable task);
  <T> Future<T> submit(Runnable task, T result);
  // ... lifecycle, bulk ops
}

submit returns a Future, which lets you:

• Wait for completion (get() blocks).
• Read the result (get() returns the Callable's value).
• Cancel the task (cancel(boolean mayInterrupt)).
• Catch the task's exception (get() re-throws it).

We cover Future and Callable in detail in the next chapter; for now, the contrast with execute is the point. execute is one-way; submit opens a return channel.

ExecutorService pool = Executors.newFixedThreadPool(4);
Future<Integer> result = pool.submit(() -> {
  // Callable<Integer>; can throw, returns a value
  return expensive();
});

Integer value = result.get();                       // waits, throws ExecutionException if task failed

Bulk operations: invokeAll and invokeAny

When you have a collection of tasks:

List<Callable<Integer>> tasks = makeTasks();

List<Future<Integer>> futures = pool.invokeAll(tasks);          // run all, wait for all
Integer first = pool.invokeAny(tasks);                          // run all, return first success, cancel the rest

invokeAll(tasks, timeout, unit) runs them but gives up after a deadline; tasks that didn't finish come back as Futures whose isDone() is true but were cancelled.

invokeAny is the right tool for redundant queries — call three DNS servers, take whichever answers first, cancel the others.

ScheduledExecutorService — delays and repeats

When you need a delay or a periodic schedule:

ScheduledExecutorService sched = Executors.newScheduledThreadPool(2);

sched.schedule(() -> log("once, after 5 seconds"), 5, TimeUnit.SECONDS);

sched.scheduleAtFixedRate(this::flush, 0, 1, TimeUnit.SECONDS);
// runs at t=0, t=1, t=2, ... — even if a run takes longer, the next one queues

sched.scheduleWithFixedDelay(this::poll, 0, 1, TimeUnit.SECONDS);
// runs at t=0, then 1 second AFTER the previous finished — back-to-back delay is what's fixed

The difference between atFixedRate and withFixedDelay is whether the period is between starts or between end and next start. For "I want to flush every second on the clock," use atFixedRate; for "I want a 1-second gap between runs no matter how long they take," use withFixedDelay.

If a scheduled task throws, the future executions are silently cancelled. The scheduler logs nothing. Always wrap scheduled tasks in a top-level try/catch so they keep running:

sched.scheduleAtFixedRate(() -> {
  try { flush(); }
  catch (Throwable t) { log.error("flush failed", t); }
}, 0, 1, TimeUnit.SECONDS);

Forgetting this is the most common scheduler bug in production Java.

Sizing the pool

The right pool size depends on what the tasks do.

For CPU-bound work, the rule of thumb is N + 1 threads on an N-core machine. Each thread keeps one core busy; the +1 covers the rare moment a thread hits a memory stall.

For I/O-bound work, the right number is much larger. The rough formula:

threads = cores * (1 + (wait_time / compute_time))

If your tasks are 90% waiting on the database, the multiplier is 10x — 80 threads on 8 cores. The exact number depends on the specific I/O pattern; profile and tune.

In practice, run two pools: a small one for CPU work and a large one for I/O. Don't mix them — a slow database call inside a CPU-pool thread blocks a core that should be computing.

Java 21 virtual threads change this math fundamentally: blocking on I/O no longer wastes a platform thread, so you can use a virtual-thread-per-task executor and stop sizing entirely. We cover that at the end of the part.

Executors factories — quick reference

The factory methods all return ExecutorService (or a sub-interface). Each is a ThreadPoolExecutor with specific knob values:

Avoid newFixedThreadPool and newCachedThreadPool for production overload paths — both have unbounded growth axes. Use new ThreadPoolExecutor(...) directly with a bounded queue.

The standard shutdown sequence

A pool that's never shut down keeps its non-daemon worker threads alive, preventing JVM exit. Every pool you create needs the same cleanup pattern:

ExecutorService pool = Executors.newFixedThreadPool(4);
try {
  // ... submit work, gather results ...
} finally {
  pool.shutdown();
  try {
    if (!pool.awaitTermination(10, TimeUnit.SECONDS)) {
      pool.shutdownNow();
      pool.awaitTermination(5, TimeUnit.SECONDS);
    }
  } catch (InterruptedException e) {
    pool.shutdownNow();
    Thread.currentThread().interrupt();
  }
}

Or, since Java 19, the same thing via try-with-resources:

try (var pool = Executors.newFixedThreadPool(4)) {
  pool.submit(...);
  pool.submit(...);
}                                                    // close() runs shutdown + awaitTermination

The Java 19 ExecutorService.close() does the polite shutdown then waits indefinitely; combine it with a watchdog if you can't afford an infinite wait.

A worked example: the framework end to end

The program below uses each of the three interfaces — Executor for fire-and-forget, ExecutorService for results, and ScheduledExecutorService for periodic — all in one.

What to take from the run:

• Section 2 used try (ExecutorService pool = ...) — the Java 19 close-on-scope-exit pattern. The pool's close() runs shutdown() then waits. That's the cleanest shutdown shape; for older code or stricter deadlines, drop back to the shutdown + awaitTermination + shutdownNow sequence.
• Section 3 ran three tasks of 50/80/20 ms on 4 workers. invokeAll returned only after the slowest finished — about 80 ms. That's the "wait for all" contract. The sum over the futures was the sum of the values they returned, in submission order.
• Section 4 ran the same shape with invokeAny. The fastest task (50 ms) returned first; the others were cancelled. invokeAny is exactly the right shape for "first successful response" patterns — DNS lookups against multiple servers, mirror downloads, latency races.
• Section 5 used scheduleAtFixedRate with a 60 ms period. Each tick fired on a scheduled-pool thread. The try/catch wrapper inside the body is the production shape — if a scheduled task throws, the scheduler silently cancels future runs. Wrapping every body in a top-level catch keeps that from happening.
• The scheduled task was explicitly cancel(false)'d before the program exited. Cancelling and shutting down the scheduler is what lets the JVM terminate; without it, the scheduler holds non-daemon threads and the program hangs. The same applies to every executor you create.

What's next

The next chapter, Java Callable and Future, dives into the result-handling side of submit — Callable<V>, Future<V>, cancellation, and the standard idioms for getting a value out of an async task.

Practice