AYOUB.*
← ENTERPRISE DISTRIBUTED ARCHITECTUREREPOS ↗

Appendix A · Part IV: Reference

Reference Tables

The numbers from all nine chapters in one place. Treat the ratios as durable and the absolutes as approximate for 2026 hardware and software.

By From Enterprise Distributed Architecture and Cloud-Native Platforms8 min read
In this chapter
  1. Transport and network
  2. Serialization and calls
  3. Cryptography
  4. Consensus and replication
  5. Consistency models
  6. Storage engines
  7. Service mesh and resilience
  8. Edge and runtimes
  9. Orchestration
  10. Observability and reliability
  11. AI infrastructure

Transport and network

Path Round trip
Loopback 20 to 50 us
Same rack 100 to 250 us
Same availability zone 250 us to 1 ms
Across zones, one region 0.5 to 2 ms
US east to US west 60 to 80 ms
US east to Europe 70 to 90 ms
US east to Asia Pacific 150 to 200 ms
Antipodal up to 280 ms

Connection setup cost, first visit and repeat visit.

Stack New Resumed Survives IP change
HTTP/1.1 over TCP and TLS 1.2 3 RTT 2 RTT no
HTTP/2 over TCP and TLS 1.3 2 RTT 1 RTT no
HTTP/2 with TCP Fast Open 1 RTT 0 RTT no
HTTP/3 over QUIC 1 RTT 0 RTT yes

Data path cost per packet, one direction.

Path Per packet Packets per second, one core
Ordinary socket write 1.5 to 3 us 300,000 to 600,000
Socket with MSG_ZEROCOPY 1.2 to 2.5 us 400,000 to 800,000
sendfile or splice 0.8 to 1.5 us 600,000 to 1,200,000
AF_XDP, zero-copy 150 to 400 ns 10,000,000 to 25,000,000
XDP drop in the driver 40 to 80 ns 15,000,000 to 30,000,000
DPDK poll mode 50 to 150 ns 10,000,000 to 15,000,000
RDMA write, end to end 1 to 2 us fabric bound

Serialization and calls

Format, 1 KB record, 20 fields Size Encode Decode all Access 2 fields
JSON, general library 1,050 B 2,400 ns 3,800 ns 3,800 ns
JSON, SIMD parser 1,050 B 1,900 ns 600 ns 600 ns
MessagePack 720 B 900 ns 1,100 ns 1,100 ns
Protocol Buffers 480 B 420 ns 560 ns 560 ns
Cap'n Proto 620 B 300 ns 0 ns 25 ns
FlatBuffers 640 B 380 ns 0 ns 20 ns
Boundary Cost
Direct call, inlined 0 ns
Virtual call, megamorphic 3 to 8 ns
Java to native, FFM API 5 to 15 ns
Go to C, cgo 40 to 100 ns
Unix socket round trip 5 to 15 us
HTTP within a datacenter 200 us to 2 ms
HTTP across regions 30 to 150 ms

Cryptography

Operation Cost Handshakes per second per core
X25519 key generation plus agreement 30 to 50 us
ECDSA P-256 sign ~25 us
ECDSA P-256 verify ~70 us
RSA-2048 sign ~700 us
RSA-2048 verify ~25 us
Full handshake, ECDSA P-256, server side 80 to 150 us 7,000 to 12,000
Full handshake, RSA-2048, server side 750 to 900 us 1,100 to 1,300
Resumption from a session ticket 10 to 25 us 40,000 to 100,000
AES-128-GCM, AES-NI 4 to 8 GB/s
AES-128-GCM, VAES and AVX-512 12 to 20 GB/s
ChaCha20-Poly1305, no AES hardware 1.5 to 3 GB/s

Post-quantum sizes, against their classical counterparts.

Object Classical Post-quantum
Key exchange public value 32 B (X25519) 1,184 B (ML-KEM-768)
Key exchange ciphertext 32 B 1,088 B
Signature public key 32 B (Ed25519) 1,952 B (ML-DSA-65)
Signature 64 B 3,309 B

Consensus and replication

Deployment, five-node group Commit latency Tolerates
One rack 0.3 to 0.6 ms 2 node failures
Three zones, one region 1.5 to 3 ms 1 zone failure
Three regions, one continent 25 to 40 ms 1 region failure
Three regions, intercontinental 90 to 160 ms 1 region failure
Five regions, global 120 to 220 ms 2 region failures

Raft timing, each level an order of magnitude apart.

Parameter Typical
Broadcast time 0.5 to 20 ms
Heartbeat interval 50 to 150 ms
Election timeout 150 to 600 ms, randomized
Snapshot interval 10,000 to 100,000 entries
Protocol Steady-state cost Leader
Single-decree Paxos 2 RTT per value none
Multi-Paxos 1 RTT to a majority stable
Raft 1 RTT to a majority stable
Flexible Paxos 1 RTT to a small quorum stable
EPaxos 1 RTT to nearest, 2 on conflict none

Consistency models

Model Cost Available under partition
Strict serializability consensus plus a total order no
Linearizability 1 RTT to a majority no
Sequential consistency total order broadcast no
Snapshot isolation a timestamp and version storage partial
Causal consistency dependency tracking yes
Read your writes sticky routing or a version token yes
Eventual consistency almost nothing yes

Isolation levels by the anomalies they permit.

Level Dirty read Non-repeatable Phantom Write skew
Read uncommitted yes yes yes yes
Read committed no yes yes yes
Repeatable read, lock based no no yes yes
Snapshot isolation no no no yes
Serializable snapshot isolation no no no no

Storage engines

Engine Read amp Write amp Space amp
B+tree, in place 1.0 to 1.3 4 to 40 1.3 to 2.0
B+tree, copy on write 1.0 to 1.3 8 to 60 1.5 to 3.0
LSM, leveled 1.0 to 1.5 10 to 30 1.05 to 1.2
LSM, tiered 2 to 6 4 to 10 1.5 to 3.0
LSM, key-value separated 1.5 to 2.5 2 to 5 1.3 to 2.0
Column store, immutable 0.1 to 0.5 1 to 2 0.1 to 0.3
Storage fsync Commits/s, one thread With group commit
Consumer SSD, no power loss protection 1 to 10 ms 100 to 1,000 no gain
Enterprise NVMe with power loss protection 20 to 100 us 10,000 to 50,000 200,000+
Network block storage, 3 replicas 0.5 to 2 ms 500 to 2,000 20,000 to 60,000

Quorum rule: W plus R greater than N gives read-write overlap. It does not repair replicas and it does not order concurrent writes.

Service mesh and resilience

Model Memory per workload Added p50 Added p99
No mesh, library 0 0 0
Envoy sidecar 50 to 150 MiB 0.5 to 2 ms 2 to 8 ms
Linkerd micro-proxy 10 to 30 MiB 0.2 to 0.8 ms 1 to 4 ms
Ambient, ztunnel only 3 to 15 MiB per node 0.2 to 0.6 ms 1 to 3 ms
eBPF datapath, no proxy under 5 MiB per node 0.05 to 0.3 ms 0.5 to 2 ms

Retry amplification: retries per hop raised to the power of the hop count. Three retries across four hops give 81 requests at the deepest service.

Queue delay in service times, single server.

Utilization Delay multiple
50 percent 1.0
70 percent 2.3
80 percent 4.0
90 percent 9.0
95 percent 19.0
99 percent 99.0

Little's law: concurrency equals arrival rate times latency. Use it to size every pool, every concurrency limit, and every bulkhead.

Edge and runtimes

Model Cold start Memory per instance
Container 100 to 500 ms 30 to 200 MiB
MicroVM, Firecracker ~125 ms 5 MiB plus the guest
V8 isolate 1 to 5 ms 2 to 10 MiB
WebAssembly module under 1 ms 1 to 5 MiB
Rate limiter State per caller Burst Boundary flaw
Fixed window 1 counter full limit instantly yes, 2x
Sliding window log 1 timestamp per request exact none
Sliding window counter 2 counters close to exact none
Token bucket tokens plus a timestamp up to capacity none
GCRA 1 timestamp up to a tolerance none

Orchestration

etcd limit Default
Database size 2 GiB, raisable to 8 GiB
Value size 1.5 MiB
Practical node count 5,000 per cluster
Practical pod count 150,000 per cluster
Isolation Start cost Run cost
Namespaces ~1 ms 0
cgroup v2 ~1 ms under 1 percent
Seccomp filter ~0.1 ms 1 to 5 percent
gVisor 100 to 300 ms 20 to 50 percent on I/O
Kata Containers 100 to 500 ms 5 to 15 percent

Block volume failover: unreachable timeout plus detach plus attach plus mount plus application start, commonly 2 to 8 minutes.

Observability and reliability

Probability that a fan-out hits at least one slow backend.

Backends Backend p99 Backend p99.9
10 9.6 percent 1.0 percent
50 39.5 percent 4.9 percent
100 63.4 percent 9.5 percent
500 99.3 percent 39.4 percent
1,000 99.996 percent 63.2 percent
Objective Downtime per 30 days Failures at 1,000 rps
99 percent 7 h 12 m 25,920,000
99.9 percent 43.2 m 2,592,000
99.95 percent 21.6 m 1,296,000
99.99 percent 4.32 m 259,200
99.999 percent 25.9 s 25,920
Burn rate Long window Short window Action
14.4 1 hour 5 minutes page immediately
6 6 hours 30 minutes page
3 1 day 2 hours ticket
1 3 days 6 hours ticket

AI infrastructure

Decode ceiling equals memory bandwidth divided by model bytes.

Model and precision Weight bytes 8 TB/s 22 TB/s
8B, BF16 16 GB ~500 tok/s ~1,375 tok/s
70B, BF16 140 GB ~57 tok/s ~157 tok/s
70B, FP8 70 GB ~114 tok/s ~314 tok/s
70B, FP4 35 GB ~228 tok/s ~628 tok/s

Key and value cache bytes per token equals 2 times layers times key-value heads times head dimension times bytes per element.

Attention scheme Bytes per token 8K context 128K context
Multi-head, 64 heads 2.6 MB 21 GB 335 GB
Grouped query, 8 groups 328 KB 2.6 GB 42 GB
Multi-query, 1 head 41 KB 0.3 GB 5.2 GB

Forward pass costs about 2 operations per parameter per token. Training costs about 6. Adam in mixed precision costs about 16 bytes per parameter before activations.

Parallelism Communication per layer Interconnect
Data one all-reduce per step ordinary network
Tensor 2 all-reduce, forward NVLink class, in node
Pipeline point-to-point activations InfiniBand or Ethernet
Expert 2 all-to-all, forward NVLink or fast fabric
Fully sharded data all-gather plus reduce-scatter fast fabric

Pipeline bubble equals p minus 1 over m plus p minus 1, for p stages and m microbatches.

Vector index, 100 million 768-dimension vectors.

Index Memory p50 Recall@10
Flat, exact 307 GB 500+ ms 1.00
HNSW, M=32 400 to 500 GB 1 to 5 ms 0.95 to 0.99
HNSW with SQ8 110 to 140 GB 2 to 6 ms 0.94 to 0.98
IVF-PQ, m=96 10 to 15 GB 3 to 10 ms 0.85 to 0.93
DiskANN on NVMe 15 to 25 GB RAM 5 to 20 ms 0.94 to 0.98

Appendix A of Enterprise Distributed Architecture and Cloud-Native Platforms, revision 1.0. Attribution and sole author credit: Ayoub Zulfiqar.