Race Conditions: The Bugs That Only Show Up In Production

A race condition is a bug whose outcome depends on the relative timing of two or more operations. The code is logically correct; the interleaving isn't. That's why it passes tests, passes staging, and blows up on Black Friday.

The canonical example: lost update

// Two users both redeem a 100-credit coupon at the same time:
const user = await findUser(id);        // T1 reads credits=100
                                         // T2 reads credits=100
user.credits -= 100;                     // T1: 0
await save(user);                        // T1 writes 0
                                         // T2: 0
                                         // T2 writes 0  → user redeemed twice!

Two read-modify-write flows interleaved. Final state: user got double credit. No exception, no stack trace — just wrong data.

Three places races love to live

1. Check-then-act. if (!exists) create(). Two threads both see "not exists," both create.
2. Read-modify-write. As above. The cure is one atomic operation, not two.
3. Initialisation. Lazy-initialised singletons that aren't synchronised correctly.

The fixes, from cheapest to strongest

• Atomic DB operation. UPDATE users SET credits = credits - 100 WHERE id = ? AND credits >= 100. One statement, the database handles the concurrency. Check rows_affected.
• Optimistic locking. Version column, retry on conflict. Perfect for contested-but-infrequent writes.
• Pessimistic lock. SELECT … FOR UPDATE. Slower but foolproof.
• Atomic primitive. In-memory: AtomicLong, sync.Mutex, channel-based ownership.
• Idempotent design. Client sends an idempotency key; server deduplicates. Converts the race into a non-issue.

How to find races you don't know about

Load test. A single-threaded benchmark cannot find them. Tools like jcstress (Java), the Go race detector (go test -race), and Jepsen-style fault injection expose races deterministically.

Mental model

Any time you read a value, compute something from it, then write it back — assume something else ran in the gap. Either push the check into the write itself, or hold a lock across both.