Appendix A · Part V: Reference
Reference Tables
Every number in this book, gathered in one place. Treat the ratios as durable and the absolutes as approximate for 2026 hardware.
In this chapter
- Latency, from a register to a continent
- Memory hierarchy sizes and bandwidth
- Cache and memory constants
- Queueing and utilization
- Probabilistic structure sizing
- Storage engine amplification
- Isolation levels and the anomalies they permit
- Network round trips
- Error budgets
- Inference sizing
- Call and boundary costs
Latency, from a register to a continent
The single most useful table in the book. Each row is one operation, and the relative column shows the cost against an L1 cache reference.
| Operation | Time | Relative | Chapter |
|---|---|---|---|
| L1 data cache reference | 1 ns | 1x | 1 |
| Atomic increment, uncontended | 6 ns | 6x | 4 |
| Branch misprediction | 5 ns | 5x | 1 |
| L2 cache reference | 3.5 ns | 4x | 1 |
| Mutex lock and unlock, uncontended | 18 ns | 18x | 4 |
| L3 cache reference | 20 ns | 20x | 1 |
| Hash table lookup, resident | 25 ns | 25x | 3 |
| Atomic increment, same socket | 45 ns | 45x | 4 |
| Main memory reference, local node | 80 ns | 80x | 1 |
| B+tree search, 512-way, resident | 95 ns | 95x | 3 |
| Main memory reference, remote node | 140 ns | 140x | 1 |
| Atomic increment, cross socket | 220 ns | 220x | 4 |
| CXL attached memory reference | 370 ns | 370x | 1 |
| System call, mitigations enabled | 400 ns | 400x | 2 |
| Kernel network stack, one direction | 2,000 ns | 2,000x | 9 |
| Context switch, same core | 2,000 ns | 2,000x | 2 |
| Futex wake, thread handoff | 5,000 ns | 5,000x | 2 |
| RDMA write, same rack | 2,000 ns | 2,000x | 9 |
fsync, enterprise NVMe |
50,000 ns | 50,000x | 7 |
| NVMe random 4 KiB read | 80,000 ns | 80,000x | 1 |
| Network round trip, same rack | 150,000 ns | 150,000x | 8 |
| Network round trip, cross zone | 1,000,000 ns | 1e6x | 8 |
| Consensus commit, three zones | 2,000,000 ns | 2e6x | 8 |
| Network round trip, transatlantic | 80,000,000 ns | 8e7x | 8 |
| Container cold start, lazy pull | 500,000,000 ns | 5e8x | 10 |
| JVM cold start, no native image | 2,000,000,000 ns | 2e9x | 5 |
Memory hierarchy sizes and bandwidth
| Level | Size | Bandwidth | Latency |
|---|---|---|---|
| Registers | 32 x 64 bit | ~1 TB/s | 0 ns |
| L1 data, per core | 32 to 64 KiB | 200 to 400 GB/s | 1.0 ns |
| L2, per core | 1 to 3 MiB | 100 to 200 GB/s | 3.5 ns |
| L3, shared | 32 to 500 MiB | 200 to 500 GB/s | 20 ns |
| DRAM, per socket | 0.5 to 3 TiB | 400 to 600 GB/s | 80 ns |
| CXL expander | 1 to 16 TiB | PCIe limited | 370 ns |
| GPU HBM3e, Blackwell class | 192 GB | 8 TB/s | ~400 cycles |
| GPU HBM4, Rubin class | 288 GB | 22 TB/s | ~400 cycles |
| NVLink 5, peer GPU | peer memory | 1.8 TB/s | ~2 us |
| PCIe 5 x16 | host memory | 64 GB/s | ~10 us |
Cache and memory constants
| Constant | Value |
|---|---|
| Cache line, x86-64 and most Arm | 64 bytes |
| Cache line, Apple silicon | 128 bytes |
| Adjacent line prefetch unit, Intel | 128 bytes |
| Page size, default | 4 KiB |
| Huge page sizes, x86-64 | 2 MiB, 1 GiB |
| Prefetcher boundary | 4 KiB page |
| Second-level TLB entries, typical | 2,048 to 4,096 |
| TLB reach, 4 KiB pages | ~8 MiB |
| TLB reach, 2 MiB pages | ~4 GiB |
| Outstanding line fills per core | 10 to 16 |
| Reorder buffer entries, 2026 core | 500 to 600 |
| Branch misprediction penalty | 15 to 20 cycles |
| SIMD widths | 128, 256, 512 bits; SVE 128 to 2048 |
Queueing and utilization
Queue delay in units of service time, single server.
| Utilization | Delay multiple | With 1 ms service time |
|---|---|---|
| 50 percent | 1.0 | 1 ms |
| 70 percent | 2.3 | 2.3 ms |
| 80 percent | 4.0 | 4 ms |
| 90 percent | 9.0 | 9 ms |
| 95 percent | 19.0 | 19 ms |
| 99 percent | 99.0 | 99 ms |
| 99.9 percent | 999.0 | 999 ms |
Little's law: concurrency equals arrival rate times latency. Use it to size every thread pool, connection pool, and concurrency limit.
Universal Scalability Law: capacity at N equals N divided by (1 + alpha(N-1) + beta N(N-1)). Peak at N equals the square root of (1 - alpha) divided by beta.
Probabilistic structure sizing
Bloom filter, bits per element and hash count by target false positive rate.
| False positive rate | Bits per element | Hashes | 1e9 elements |
|---|---|---|---|
| 10 percent | 4.8 | 3 | 600 MB |
| 1 percent | 9.6 | 7 | 1.2 GB |
| 0.1 percent | 14.4 | 10 | 1.8 GB |
| 0.01 percent | 19.2 | 13 | 2.4 GB |
| 0.001 percent | 24.0 | 17 | 3.0 GB |
HyperLogLog: 16,384 registers of 6 bits equals 12 KB, with 0.81 percent standard error, for any cardinality.
Count-Min sketch: width equals e divided by the error bound, depth equals the natural log of one over the failure probability. Never underestimates.
Storage engine amplification
| Engine | Read amp | Write amp | Space amp |
|---|---|---|---|
| B+tree, in place | 1.0 to 1.3 | 4 to 40 | 1.3 to 2.0 |
| B+tree, copy on write | 1.0 to 1.3 | 8 to 60 | 1.5 to 3.0 |
| LSM, leveled | 1.0 to 1.5 | 10 to 30 | 1.05 to 1.2 |
| LSM, tiered | 2 to 6 | 4 to 10 | 1.5 to 3.0 |
| LSM, key-value separated | 1.5 to 2.5 | 2 to 5 | 1.3 to 2.0 |
| Column store, immutable | 0.1 to 0.5 | 1 to 2 | 0.1 to 0.3 |
Isolation levels and the anomalies they permit
| Level | Dirty read | Non-repeatable | Phantom | Write skew | Lost update |
|---|---|---|---|---|---|
| Read uncommitted | yes | yes | yes | yes | yes |
| Read committed | no | yes | yes | yes | yes |
| Repeatable read, lock based | no | no | yes | yes | no |
| Snapshot isolation | no | no | no | yes | no |
| Serializable snapshot isolation | no | no | no | no | no |
| Serializable, two-phase locking | no | no | no | no | no |
Network round trips
| Path | Round trip |
|---|---|
| Loopback | 20 to 50 us |
| Same rack | 100 to 250 us |
| Same availability zone | 250 us to 1 ms |
| Across zones, one region | 0.5 to 2 ms |
| US east to US west | 60 to 80 ms |
| US east to Europe | 70 to 90 ms |
| US east to Asia Pacific | 150 to 200 ms |
| Antipodal | up to 280 ms |
Error budgets
| Objective | Downtime per 30 days | Failed requests at 1,000 per second |
|---|---|---|
| 99 percent | 7 h 12 m | 25,920,000 |
| 99.9 percent | 43.2 m | 2,592,000 |
| 99.95 percent | 21.6 m | 1,296,000 |
| 99.99 percent | 4.32 m | 259,200 |
| 99.999 percent | 25.9 s | 25,920 |
Multi-window burn rate alerting: 14.4 over 1 hour pages, 6 over 6 hours pages, 3 over 1 day tickets, 1 over 3 days tickets. Pair each long window with a short one at one twelfth its length.
Inference sizing
Key and value cache bytes per token equals 2 times layers times key-value heads times head dimension times bytes per element.
Decode rate ceiling at batch size one equals memory bandwidth divided by model bytes.
Forward pass cost equals about 2 operations per parameter per token. Training cost equals about 6.
Training memory with Adam in mixed precision equals about 16 bytes per parameter, before activations.
| Parallelism | Communication per layer | Interconnect |
|---|---|---|
| Data | one all-reduce per step | ordinary network |
| Tensor | two all-reduce, forward | NVLink class, in node |
| Pipeline | point-to-point activations | InfiniBand or Ethernet |
| Expert | two all-to-all, forward | NVLink or fast fabric |
| Sequence | ring exchange of keys and values | NVLink class |
Call and boundary costs
| Boundary | Cost |
|---|---|
| Direct call, inlined | 0 ns |
| Virtual call, monomorphic | 0.9 ns |
| Virtual call, megamorphic | 3 to 8 ns |
| Java to native, FFM API | 5 to 15 ns |
| Java to native, JNI | 15 to 40 ns |
| Go to C, cgo | 40 to 100 ns |
| Python to C, C API | 60 to 150 ns |
| Unix socket round trip | 5 to 15 us |
| HTTP within a datacenter | 200 us to 2 ms |
| HTTP across regions | 30 to 150 ms |