performance-profiling

Performance Profiling

A structured approach to diagnosing and fixing performance problems. The workflow is the same across languages; the tools differ.

Rule zero: Profile before optimising. Optimising without data is guessing. Guessing wastes time and often makes things worse.

Workflow

1. Define the problem

Write one sentence: "The GET /users endpoint takes 4s at p95 under 50 concurrent users; the target is < 500ms."

Without a measurable baseline and a concrete target, you won't know if your optimisation worked.

2. Form a hypothesis

Based on the symptoms, guess the likely cause:

• Slow endpoint → database query (N+1, missing index, large result set)?
• High CPU → tight loop, regex, serialisation?
• High memory → unbounded cache, large object held in scope, leak?
• Slow startup → heavy imports, unnecessary initialisation?

3. Profile

Pick the appropriate tool (see below) and run it against your hypothesis. Look at the top 5–10 most expensive functions/frames.

4. Benchmark

Measure before you change anything. Write a reproducible benchmark:

# HTTP endpoint (Apache Bench)
ab -n 1000 -c 50 http://localhost:3000/users

# Or hey (Go-based, better output)
hey -n 1000 -c 50 http://localhost:3000/users

Record p50, p95, p99 and throughput. This is your baseline.

5. Fix and re-benchmark

Make one change at a time. Re-run the benchmark and compare to the baseline. Multiple simultaneous changes make it impossible to know which one helped.

6. Verify under realistic load

Profiling under idle or synthetic load can miss real bottlenecks. Use production-shaped data volumes and concurrency levels.

Python

cProfile (deterministic, low overhead)

import cProfile
cProfile.run('my_function()', sort='cumulative')

Or on the command line:

python -m cProfile -s cumulative my_script.py | head -30

Key columns: tottime (time in this function only) and cumtime (including all called functions). Focus on functions with high tottime or unexpectedly high call counts.