Java JIT Compilation

Java is famous for "compile once, run anywhere," but that is only half the story. The javac compiler turns your source into bytecode, not native machine code, and the JVM starts out interpreting that bytecode one instruction at a time. The piece that makes Java fast is the JIT (Just-In-Time) compiler: while your program runs, the JVM watches which methods get called the most and compiles those "hot" methods into optimized native code on the fly.

This chapter explains how the two-stage compilation model works, what HotSpot's tiered compiler does, and why a Java program speeds up the longer it runs.

Two compilers, two jobs

There are really two compilers in the Java world, and confusing them is a common beginner mistake.

javac runs once and produces platform-independent bytecode. The JIT lives inside the running JVM and produces CPU-specific machine code tailored to the exact processor you are on. That is why the same .jar runs everywhere yet can still reach near-native speed.

// Build time: javac Hello.java  ->  Hello.class (bytecode)
// Run time:   java Hello        ->  JVM interprets, then JIT-compiles hot methods
public class Hello {
    public static void main(String[] args) {
        System.out.println("Bytecode now, native code soon.");
    }
}

Interpreter first, then JIT

When a method first runs, the JVM interprets it: there is no compilation cost, so startup is fast, but each bytecode is slow to execute. The JVM keeps a per-method invocation counter (and a back-edge counter for loops). Once a method is called often enough to cross a threshold, the JVM hands it to the JIT to be compiled into native code, and future calls jump straight into that fast version.

This is why a long-running server gets faster after warm-up: the methods on its hot path eventually get compiled, while rarely used code stays interpreted (so no compilation effort is wasted on it).

// 'process' is on the hot path. After enough calls it gets JIT-compiled;
// 'logRareError' may stay interpreted forever because it almost never runs.
void handleRequest(Request r) {
    process(r);                 // hot: many invocations -> compiled
    if (r.isMalformed()) {
        logRareError(r);        // cold: rarely called -> stays interpreted
    }
}

Tiered compilation: C1 and C2

Modern HotSpot uses tiered compilation, which blends two JIT compilers so you get fast startup and peak performance:

• C1 (the client compiler) compiles quickly with light optimization. It gets hot methods to native code fast and inserts profiling counters.
• C2 (the server compiler) compiles more slowly but optimizes aggressively, using the profile C1 gathered (inlining, loop unrolling, escape analysis, dead-code elimination).

A method climbs through tiers as it gets hotter:

Because C2 optimizes based on observed behavior, it can make bets the static javac compiler never could — for example, inlining a virtual call because in practice only one implementation ever shows up.

// C2 can speculatively inline this even though 'pay' is virtual,
// because profiling showed every call so far used CreditCard.
abstract class Payment { abstract void pay(int cents); }
class CreditCard extends Payment { void pay(int cents) { /* ... */ } }

void checkout(Payment p) {
    p.pay(1999);   // megamorphic in theory; monomorphic in practice -> inlined
}

Deoptimization: undoing a bet

Speculative optimizations can turn out wrong. If C2 inlined CreditCard.pay and then a PayPal object finally arrives, the optimized code is no longer valid. HotSpot handles this with deoptimization: it discards the bad native code, falls back to the interpreter for that method, and may later recompile with the new information. This safety net is what lets the JIT optimize aggressively without ever producing wrong results.

// First 100000 calls: only CreditCard -> C2 inlines aggressively.
// Call 100001 passes a PayPal -> the assumption breaks ->
//   HotSpot deoptimizes, reverts to interpreter, and recompiles later.
checkout(new CreditCard());
checkout(new PayPal());   // triggers deoptimization of the inlined version

Watching the tiers with a runnable example

A real warm-up benchmark needs millions of loop iterations, which a sandbox cannot run. Instead, the program below models the promotion decision HotSpot makes — classifying a method by how many times it has been invoked against the default tier thresholds — and reads genuine JIT facts from the running JVM through CompilationMXBean. Run it and watch a method move from interpreted, to C1, to C2 as its call count climbs.

What to take from the run:

• The JVM reports itself as the HotSpot 64-Bit Tiered Compilers, confirming both C1 and C2 are active in a normal java launch.
• Methods called only 1 or 500 times stay at Tier 0 (interpreted) — the JIT does not waste effort on cold code.
• Crossing the 2000 threshold promotes the method to Tier 3 (C1 compiled), the fast-to-produce native version that also profiles.
• Crossing 10000 (and 100000) promotes it to Tier 4 (C2), the fully optimized code that delivers peak speed.
• CompilationMXBean.getTotalCompilationTime() exposes real JIT activity from inside Java, proving compilation happens while the program runs, not ahead of time.

Practice