High-throughput APIs: what I learned

An API serving 100 requests a minute and one serving 100,000 are two different animals. Making that transition is where I learned that most bottlenecks aren't where you first look.

##Measure, don't guess

The first instinct is always to optimize the code. But a profiler will tell you in ten minutes that 80% sits in a single N+1 query.

##Cache is not a silver bullet

Cache invalidation is one of the two hard problems in computing. Before you cache, ask: is stale data actually acceptable here?

EXPLAIN ANALYZE
SELECT * FROM orders
WHERE user_id = $1
ORDER BY created_at DESC
LIMIT 20;
-- Seq Scan → 240ms.  after adding an index → 1.8ms

One good index is sometimes worth more than a week of refactoring. Always look at the query plan.