Nine spreads on how real systems handle more users, more data and more failures, with the numbers and trade-offs behind each choice.
"Stateless services behind a load balancer, a cache for hot reads, a replicated sharded database, a queue for slow work, and a CDN at the edge."
Use the tabs or the ← → keys.
Rough math tells you whether you need one server or a thousand.
| Latency | Roughly |
|---|---|
| CPU cache / RAM read | 1 ns / 100 ns |
| SSD random read | 100 µs |
| same-datacenter round trip | 0.5 ms |
| cross-continent round trip | 150 ms |
Example: a photo app, 10 M daily users, 2 uploads each, 2 MB per photo writes = 20 M / 86,400 s ≈ 230 uploads/s (peak ×3 ≈ 700/s) reads = 100× writes ≈ 23,000 reads/s storage = 20 M × 2 MB × 365 ≈ 14.6 PB / year → object storage + CDN rule: 1 day ≈ 10^5 s · 1 M/day ≈ 12/s · ×2–3 for peak
"Before I choose anything: peak QPS, data size, read-to-write ratio, and the availability target."
A server that keeps no per-user state can be cloned freely. Put sessions in Redis or a signed token, files in object storage, and the LB can send any request to any server and drop a dead one.
# nginx as an L7 load balancer
upstream app {
least_conn;
server 10.0.0.11:3000 max_fails=3 fail_timeout=10s;
server 10.0.0.12:3000 max_fails=3 fail_timeout=10s;
server 10.0.0.13:3000 backup;
}
server {
listen 443 ssl;
location / { proxy_pass http://app; proxy_set_header X-Forwarded-For $remote_addr; }
}
/health and removes failing nodes"Scale out stateless tiers behind an L7 balancer with health checks and autoscaling."
// cache-aside with Redis (Node)
async function getUser(id) {
const key = `user:${id}`;
const hit = await redis.get(key);
if (hit) return JSON.parse(hit); // fast path
const user = await db.one('SELECT * FROM users WHERE id=$1', [id]);
await redis.set(key, JSON.stringify(user), { EX: 300 + Math.floor(Math.random() * 60) }); // TTL + jitter
return user;
}
async function updateUser(id, data) {
await db.query('UPDATE users SET ... WHERE id=$1', [id]);
await redis.del(`user:${id}`); // invalidate on write
}
| Write policy | Behaviour |
|---|---|
| cache-aside | app fills on miss (most common) |
| write-through | write cache and DB together; consistent, slower writes |
| write-back | write cache, flush later; fast, risk of loss |
Cache-Control: public, max-age=31536000, immutable on hashed assets"Cache-aside with TTL jitter, invalidate on write, and a CDN for everything static."
| Store type | Good for | Examples |
|---|---|---|
| relational | money, orders, relations | PostgreSQL, MySQL |
| key-value | sessions, cache | Redis, DynamoDB |
| document | flexible records | MongoDB |
| wide-column | huge write volume | Cassandra |
| search | full-text, filters | Elasticsearch |
| object | files, media | S3 |
-- route by shard key shard = hash(user_id) % N -- simple, but changing N moves almost everything -- consistent hashing moves only ~1/N of keys when you add a node
"Replicas for reads, shard by user_id only when a single primary can't keep up, and consistency chosen per feature."
// producer: enqueue and return immediately (BullMQ + Redis)
await emailQueue.add('welcome', { userId }, { attempts: 5, backoff: { type: 'exponential', delay: 2000 } });
res.status(202).json({ queued: true });
// worker: idempotent, safe to run twice
new Worker('email', async (job) => {
if (await alreadySent(job.id)) return; // dedupe
await sendEmail(job.data.userId);
await markSent(job.id);
}, { concurrency: 10 });
"Return 202, queue the work, make consumers idempotent, and watch queue depth and age."
// token bucket in Redis (Lua keeps it atomic), sketch
local tokens = tonumber(redis.call('GET', KEYS[1]) or ARGV[1]) -- capacity
local now = tonumber(ARGV[3])
local last = tonumber(redis.call('GET', KEYS[2]) or now)
tokens = math.min(ARGV[1], tokens + (now - last) * ARGV[2]) -- refill
if tokens < 1 then return 0 end
redis.call('SET', KEYS[1], tokens - 1); redis.call('SET', KEYS[2], now); return 1
?after=abc123&limit=50: stable and fast, unlike deep OFFSET"Rate-limit at the gateway, cursors for paging, idempotency keys for writes, and timeouts with circuit breakers on every call out."
| Term | Plain meaning |
|---|---|
| RPO | how far back you restore to |
| RTO | how long to be back up |
| MTTR | average time to recover (the one to drive down) |
| error budget | allowed unreliability; spend it on shipping faster |
1 − 0.999 = 0.1% budget ≈ 43 min/month at a 99.9% SLO two zones at 99.9% each, either one enough → ≈ 99.9999% in theory (ignoring shared faults)
"Multi-zone, graceful degradation, canary deploys, and tested restores against explicit RPO and RTO."
# Prometheus alert: error rate above 2% for 5 minutes
- alert: HighErrorRate
expr: sum(rate(http_requests_total{status=~"5.."}[5m])) / sum(rate(http_requests_total[5m])) > 0.02
for: 5m
labels: { severity: page }
annotations: { summary: "5xx rate above 2%" }
"Golden-signal dashboards, trace IDs end to end, alerts on SLO burn, and a runbook for each page."
POST /v1/links { "url": "https://long.example/very/long/path" } → 201 { "short": "aZ3k9Q" }
GET /aZ3k9Q → 302 Location: https://long.example/…
CREATE TABLE links (
short_id CHAR(7) PRIMARY KEY,
long_url TEXT NOT NULL,
owner_id BIGINT,
created_at TIMESTAMPTZ DEFAULT now(),
expires_at TIMESTAMPTZ
);
"I'll confirm requirements, estimate the load, sketch read and write paths, then deep-dive the bottleneck and name the trade-offs."