1. Vertical vs Horizontal Scaling & Latency Numbers

The Limits of Vertical Scaling

Scaling vertically (scale-up) means adding more CPU cores, RAM, or NVMe storage to a single physical machine. It requires zero distributed coordination and eliminates network hops, but it hits strict hardware ceilings, exponential cost curves, and creates a single point of failure (SPOF).

• •Scale-up: Fast, simple, bounded by bus bandwidth and memory sockets.
• •Scale-out: Adds commodity nodes in parallel behind load balancers.
• •Shared-nothing architecture: Nodes communicate exclusively through network messages.

Latency Numbers Every Programmer Must Know

L1 cache references take ~1ns, main memory access takes ~100ns, reading 1MB sequentially from an NVMe SSD takes ~10,000ns (10µs), while a datacenter round-trip takes ~500,000ns (0.5ms) and a cross-continent network round-trip takes ~150,000,000ns (150ms). Network I/O is four to six orders of magnitude slower than CPU operations.

Back-of-the-Envelope Capacity Estimations

Designing scalable architectures requires rough sizing calculations up front: converting Daily Active Users (DAU) to Queries Per Second (QPS), calculating read/write throughput ratios, estimating daily storage ingress, and sizing bandwidth for peak spikes (typically 2x to 5x average load).

// 100M DAU with 10 actions/day -> 1 Billion queries/day
const dailyQueries = 100_000_000 * 10;
const secondsInDay = 86_400;
const averageQps = Math.round(dailyQueries / secondsInDay); // ~11,574 QPS
const peakQps = averageQps * 2; // ~23,148 QPS