System Design — Engineering Journal

Jul 02, 20268 min

Token Bucket vs Leaky Bucket: The Rate Limiter That Let the Burst Through

A rate limiter rejected traffic smoothly in every test and then let a client through at ten times its contracted rate during a real burst, taking down a downstream dependency that had no defenses of its own. The limiter was not broken. It was a token bucket, correctly implemented, doing exactly what a token bucket is designed to do, which turned out to be the wrong algorithm for a downstream that could not absorb bursts at all.

System DesignRate Limiting

Read

System Design

Jun 26, 20268 min

The TTL We Forgot: Why Traffic Kept Hitting a Server We'd Already Killed

We moved a service to a new host, repointed the DNS record, watched traffic shift over, and shut the old box down. Then the error rate climbed and stayed up for almost an hour, because a meaningful slice of clients were still sending requests to an IP that no longer answered. Nobody had ever looked at the record's TTL.

NetworkingDNS

Read

System Design

Jun 23, 20268 min

Timeout Budgets: Why One Slow Dependency Blows Your Whole Request

The checkout endpoint had a 30-second timeout. So did the payment service it called, and the fraud check that called, and the database underneath that. When one of them got slow, the timeouts stacked instead of capping, and a single sluggish dependency held thousands of connections open until the whole chain fell over.

System DesignReliability

Read

System Design

Jun 21, 20269 min

Retry Storms: How Well-Meaning Retries Turn a Blip Into an Outage

A downstream service hiccupped for two seconds. Every caller retried three times. Those retries tripled the load on a service that was already struggling, and the blip became a 20-minute outage. Backoff without jitter made it worse, not better.

System DesignResilience

Read

System Design

Jun 17, 20269 min

Head-of-Line Blocking: Why HTTP/2 Didn't Fix What You Think, and HTTP/3 Did

HTTP/2 multiplexes streams over one connection, so people assume one slow response can't block the others. It can, just one layer down, in TCP. Here is the head-of-line blocking that survived HTTP/2 and why HTTP/3 had to abandon TCP entirely to kill it.

System Design

Read

System Design

Jun 16, 20269 min

TIME_WAIT and the TCP Handshake: Why Your Server Runs Out of Ports

Connection exhaustion under load almost never means you ran out of memory — it means you ran out of ephemeral ports to TIME_WAIT sockets. Here is the TCP lifecycle that explains it and the fixes that actually work.

System Design

Read

System Design

May 14, 20269 min

Rate Limiting: Token Bucket, Leaky Bucket, and Sliding Window

Every public API needs a rate limiter, but the algorithm you pick changes the burst behaviour your users feel. A practical comparison with Redis implementations.

System DesignScaling

Read

System Design

May 12, 202610 min

The Saga Pattern: Distributed Transactions Without Two-Phase Commit

When a business operation spans several services, you can't wrap it in one ACID transaction. Sagas trade atomicity for compensations — here's how to get them right.

System DesignMicroservices

Read

System Design

Feb 08, 20269 min

The Outbox Pattern: Atomic DB and Queue Writes

You can't atomically commit to Postgres and Kafka. The outbox pattern is how senior backends finesse that without losing events.

System DesignMessaging

Read

System Design

Dec 05, 20258 min

Idempotency Keys: Making APIs Safe to Retry

Networks fail, clients retry, and one charge becomes two. Idempotency keys turn that from a postmortem into a non-event.

System DesignAPIs

Read

More topics