Programming Language Pragmatics and Memory Models

What a runtime has to decide

Every language runtime answers four questions. Who frees memory? How does a call find its target? When does source become machine code? What does one thread see of another thread's writes? The answers determine more about a service's behavior under load than the application code does.

The answers also trade against each other. A tracing garbage collector removes a whole class of bug and adds a pause. Ahead-of-time compilation removes the warm-up and loses the profile data that a just-in-time compiler uses. Ownership tracking removes both the collector and the pause and adds a constraint that the programmer must satisfy. No answer dominates, and a team that knows the trade chooses better than a team that argues from taste.

Read the cold start column together with the deployment model. A service that runs for weeks does not care about a 3 second start. A function that scales from zero on every request cares about nothing else. Chapter 10 and Chapter 16 both return to this.

Manual memory, RAII, and ownership

Manual memory management gives the programmer malloc and free. It is the fastest model when it is correct and it produces the two most exploited bug classes in software: the use after free and the buffer overflow. Roughly 70 percent of the security vulnerabilities that Microsoft and Google report in large C and C++ code bases are memory safety defects.

Resource acquisition is initialization, or RAII, ties a resource to an object lifetime. A C++ destructor runs when the object leaves scope, and it runs on the normal path and on the exception path alike. The technique generalizes past memory to file descriptors, locks, and transactions. It is deterministic, it costs nothing at run time, and it does not prevent a dangling reference to an object that already went out of scope.

Rust closes that gap with ownership and borrowing. Every value has exactly one owner. A value moves when it is assigned, and the previous owner can no longer use it. A borrow creates a reference with a lifetime that the compiler tracks, and the compiler rejects any program where a reference outlives its referent. The rule set also forbids one mutable borrow alongside any other borrow, which removes data races in safe code as a side effect.

fn main() {
    let data = vec![1, 2, 3];
    let moved = data;          // ownership moves out of `data`
    // println!("{:?}", data); // compile error: value used after move

    let mut counter = 0;
    let r1 = &counter;         // shared borrow
    // let r2 = &mut counter;  // compile error: mutable borrow while shared
    println!("{}", r1);
    counter += 1;              // fine, r1 is no longer live
    println!("{}", counter);
}

The cost of the model is a learning curve and a set of patterns that the checker cannot prove, above all cyclic data structures and self-referential types. Rc and RefCell move those checks to run time. unsafe blocks remove them and concentrate the risk in a small, auditable region.

The model moved from an argument into policy. CISA and the FBI asked software makers that serve critical infrastructure to publish a memory safety roadmap by January 1, 2026. NSA and CISA published joint guidance on adopting memory-safe languages in 2025. Linux kernel maintainers ended the experimental status of Rust in the kernel in December 2025, and Rust drivers now ship in Android, in file systems, and in NVIDIA's Nova GPU driver.

Reference counting

Reference counting frees an object when its count reaches zero. The cost is distributed rather than batched, which gives predictable latency and a steady overhead on every assignment.

CPython counts references on every object and adds a cycle collector for the cases that counting cannot free. Swift uses automatic reference counting, or ARC, which the compiler inserts at compile time and optimizes where it can prove a count is redundant. Rust offers Rc for single-threaded sharing and Arc for cross-thread sharing, and neither is the default.

Two costs appear in production. The first is atomicity. A count that several threads touch must use an atomic increment, and Chapter 4 priced that at 45 nanoseconds across cores and 220 nanoseconds across sockets. Arc::clone in a hot loop is a cache line ping-pong with a friendly name. The second is the cascade. Dropping the root of a large object graph frees every node in one call, which produces a pause that looks exactly like a collector pause.

Cycles are the correctness problem. Two objects that reference each other keep each other alive forever. CPython runs a generational cycle collector to find them. Swift does not, and it asks the programmer to break cycles with weak or unowned references. A retain cycle in a Swift application is a leak that no tool reports until memory runs out.

Tracing garbage collection

A tracing collector starts from a set of roots, which are the stacks, the registers, and the static fields. It marks every object reachable from a root and reclaims everything else. The core insight is that it never looks at garbage, so its cost scales with the live set and not with the allocation rate.

The generational hypothesis states that most objects die young. Measurements across many workloads put the figure above 90 percent. A generational collector therefore splits the heap into a young generation and an old one, collects the young generation often and cheaply, and promotes survivors.

A copying collector for the young generation makes allocation almost free. Allocation is a pointer bump in a contiguous region. A minor collection copies the few survivors out and resets the pointer, so the cost is proportional to the survivors rather than to the region size. This is why allocation in Java and Go costs about 10 to 20 nanoseconds and allocation with a general purpose malloc costs 30 to 100.

  young generation                       old generation
  +--------+--------+--------+           +------------------------+
  |  eden  | surv 0 | surv 1 |           |  promoted objects      |
  +--------+--------+--------+           +------------------------+
     ^ bump pointer allocation                 |
                                               | old -> young reference
  A minor GC must find every old -> young      v
  reference without scanning the old gen. +--------+
                                          | obj X  |  <-- lives in eden
  WRITE BARRIER                           +--------+
  every reference store runs a few instructions that record the
  containing card (512 B region) in a card table, or add the field
  to a remembered set. Minor GC scans only dirty cards.

  Cost: 1 to 5 extra instructions per reference store, always paid.

The write barrier is the price of generational collection. Every store of a reference runs a short piece of code that records the store so that the collector can find old-to-young references without scanning the whole old generation. The barrier costs a few instructions per reference store and it is paid whether or not a collection ever runs.

Concurrent collectors run the marking phase while the application runs, which turns a long pause into a short one plus a throughput cost. Concurrency creates a correctness problem: the application can hide a reference from the marker by moving it. Collectors solve it with the tricolor invariant, which forbids a black object from pointing to a white object with no gray path, and they enforce it with a barrier.

ZGC deserves a closer look because it shows how far the design moved. It stores metadata inside the pointer itself, using bits that a 64 bit address space leaves free, and it calls those colored pointers. Every load of a reference passes through a load barrier that reads the color bits and repairs the pointer if the object moved. Marking and relocation both run concurrently, and the only stop-the-world work is a root scan. Pause time therefore depends on the number of roots rather than on the heap size, which is why a 16 terabyte heap pauses for under a millisecond. JDK 21 added a generational mode and JDK 23 made it the default for ZGC.

Go took a different path. Its collector is concurrent, non-generational, and non-moving. It does not move objects, so it needs no read barrier and interior pointers stay valid, which matters because Go exposes pointers into the middle of objects. It is not generational, so every cycle scans the whole live set. The design keeps pauses under a millisecond and pays for it in collector CPU, and GOGC controls the trade by setting the heap growth that triggers a cycle.

Arena and region allocation sidesteps the whole argument for the workloads where it fits. Allocate everything for one request from a bump allocator and free the whole arena at the end of the request. Allocation costs a pointer increment, deallocation costs one pointer reset, and no collector runs. Zig, Odin, and modern C++ with pmr allocators make this a first class pattern, and the Rust bumpalo crate provides it. The constraint is that nothing can outlive the arena, which fits a request handler and does not fit a cache.

Compilation: ahead of time, just in time, and both

An ahead-of-time compiler produces machine code before the program runs. It sees the whole program, or at least one translation unit, and its compile time is not bounded. It does not know which branch will be hot, which type a virtual call will see, or which values will be constant at run time.

A just-in-time compiler produces machine code while the program runs. It starts in an interpreter, counts executions, and compiles the hot methods. It knows exactly which branch is hot and which type a call site sees, and it uses that knowledge for speculative optimization.

HotSpot runs a tiered pipeline. Tier 0 interprets. Tiers 1 through 3 use the C1 compiler with increasing profiling. Tier 4 uses C2, which performs aggressive inlining, escape analysis, loop transformation, and vectorization. A method reaches C2 after roughly 10,000 invocations by default. Until then the service runs at a fraction of its steady-state speed.

Speculation needs an escape hatch. A JIT compiler inlines a call because only one type appeared so far. It must then handle the moment when a second type appears. Deoptimization rewrites the stack frame back into a form that the interpreter understands and resumes there. The mechanism is what makes speculation safe, and a method that deoptimizes repeatedly costs more than one that was never optimized.

  relative
  throughput
   1.0 |                 ______________________ JIT, tier 4 (C2)
       |            ____/
   0.8 |  __________________________________ AOT, native, PGO
       | /     ____/
   0.6 |/     /
       |     /  ________________________ AOT, native, no PGO
   0.4 |    / _/
       |   _//
   0.2 |  //     ___ interpreter + C1
       | //_____/
   0.0 +------------------------------------------------> time
       0    100ms    1s       10s      60s

  AOT wins the first second. JIT wins after warm-up.
  Profile-guided AOT closes most of the gap and keeps the fast start.

Profile-guided optimization, or PGO, gives an ahead-of-time compiler the profile that a JIT compiler collects for free. Run the program under a representative load, record the branch and call site behavior, and recompile with the profile. Reported gains run between 5 and 20 percent for Go and for C++, and the difficulty is operational rather than technical: the profile must match production, and it goes stale.

GraalVM native image compiles a Java program to a native executable under a closed-world assumption. It performs points-to analysis over the whole program, removes unreachable code, and runs static initializers at build time. Start-up falls from hundreds of milliseconds to about ten, and memory falls by about half. The costs are a long build, a peak throughput below a warmed-up C2, and a configuration burden for reflection, dynamic proxies, and resource loading.

Dispatch and the cost of a call

A static call jumps to a known address. The compiler can inline it, and inlining is the optimization that enables every other optimization, because it exposes the callee body to constant propagation and to dead code elimination.

A virtual call reads a function pointer from a table that the object header names. The indirect branch predictor usually predicts it correctly, so the cost is one extra load plus a predicted indirect branch, which is about 2 to 4 cycles. The real cost is that the compiler cannot inline it without knowing the target.

Inline caching recovers the inlining. The JIT compiler records which type reached a call site. A monomorphic site, which has seen one type, compiles to a type test and an inlined body. A bimorphic site compiles to two tests. A megamorphic site, which has seen many types, falls back to a table lookup and loses the inlining. The practical consequence is that an interface with two implementations is fast and the same interface with twelve implementations at one call site is slow.

Generics split the same way. C++ templates and Rust generics monomorphize, which means that the compiler emits one specialized copy per type. Calls are static and inlining works, and the binary grows. Java generics use erasure, which means that one implementation handles every type and values of primitive type get boxed. Project Valhalla aims to remove the boxing with value classes and specialized generics, and parts of it are landing across recent JDK releases.

The foreign function interface row deserves attention, because teams underestimate it by an order of magnitude. A cgo call costs about as much as a hundred Go function calls, because the runtime must move the goroutine onto a system stack and tell the scheduler that the thread is blocked. A loop that crosses the boundary per element runs at the speed of the boundary. Batch the crossing: pass a slice, not an element.

Language memory models

Chapter 4 covered the hardware side of memory ordering. Each language layers its own model on top, and the models differ in what they guarantee for racy programs.

The Java memory model defines a happens-before relation and gives strong guarantees even for programs with data races. A racy read returns some value that some thread wrote, and never an invented value. Final fields carry a freeze guarantee, so a properly constructed immutable object is safe to publish through a data race. That guarantee is what makes String safe to share.

The C++ memory model declares a data race to be undefined behavior. A racy program has no defined meaning at all, and the compiler is free to assume that races do not happen. Rust makes the same choice and then uses the type system to make safe code race-free by construction, so the undefined behavior stays inside unsafe blocks.

The Go memory model is close to release and acquire. A send on a channel happens before the corresponding receive completes. A sync.Mutex unlock happens before a later lock. Go documents that a racy program is undefined and ships a race detector that finds races at run time with about a 5 to 10 times slowdown.

Choosing a runtime for a service

The decision follows the constraint that binds. Table 5.4 gives the mapping that holds in practice.

The table leaves out the factor that dominates most real decisions, which is the team. A service written well in a familiar language beats a service written badly in the theoretically correct one. Use the table when the constraint is hard and use the team when it is not.

Sources and further reading

Richard Jones, Antony Hosking, and Eliot Moss, "The Garbage Collection Handbook", second edition, for the collectors. The ZGC and Shenandoah papers and the OpenJDK wiki pages for the load barrier designs. Aleksey Shipilev's writing and the jcstress harness for the Java memory model in practice. The Rustonomicon for the ownership model at its edges. Hans Boehm, "Threads Cannot Be Implemented As a Library", 2005, for why the language needs a memory model at all. PEP 703 for the free-threaded CPython design.