python-profiling

Python Profiling & Optimization

A structured, measurement-driven workflow for making Python code faster and leaner. The core principle: never optimize without measuring first, and never trust an optimization without measuring after.

Before you start

Read references/tools-cheatsheet.md for detailed command syntax for each profiling tool. It covers installation, usage patterns, and output interpretation.

Phase 1: Understand the project

Before profiling anything, gather context:

1. Python version — Check pyproject.toml for requires-python and target-version. This determines which tools and stdlib features are available.
2. Package manager — Look for uv.lock, poetry.lock, Pipfile.lock, or requirements.txt to determine how dependencies are managed.
3. Existing benchmarks — Search for pytest-benchmark, benchmark directories, or profiling scripts already in the project.
4. Test suite — Understand how tests run so you can validate correctness after each optimization.

Phase 2: Establish a baseline

You cannot improve what you haven't measured. Before any optimization:

If the project has pytest-benchmark

Save a named baseline snapshot:

uv run pytest <benchmark_file> -m slow \
    --benchmark-only --benchmark-disable-gc \
    --benchmark-save=baseline

If the project lacks benchmarks

Create a minimal benchmark file targeting the code to optimize. Use pytest-benchmark with pedantic() for stable, reproducible results:

@pytest.mark.slow()
class TestPerformance:
    def test_hot_path(self, benchmark):
        # Setup outside the measured region
        obj = create_object()
        benchmark.pedantic(obj.hot_method, rounds=10, iterations=1000)

Use pedantic() over the simple benchmark() call — it gives explicit control over rounds and iterations, producing more stable measurements with lower variance.

Quick ad-hoc baseline (no benchmark framework)

For quick exploration before setting up proper benchmarks:

import cProfile
cProfile.run('function_to_profile()', sort='cumulative')