choosing-what-to-optimize

Choosing what to optimize

Everything here was paid for. The figures are from one pass over this repo's own test machinery, and every one of them carries the command that re-derives it, because a number in a document is a number that rots.

The short version: the repo has no cheap 20% lying around. What it has is a handful of suites doing real work against real git and real browsers. So the value of an optimization pass here is mostly in not doing the wrong one.

Measure the thing; do not grep for it

Three findings in one session were reported wrong, each because a grep stood in for a run:

claimed	actual	why the grep lied
38 subprocess calls in a suite	453 child processes	the grep counted textual occurrences; most calls span lines and most spawns come from the code under test
1 tool without a test suite	3	all three printed a pointer containing --selftest, so the grep matched the word and missed the absence
0 section markers in an 8000-line file	42	the lint's own regex allows two spaces of indentation; the grep was anchored at column 0

The third was the expensive one: it was the premise of a plan. The lint would have passed on that file, so the rule being added to force a split would have forced nothing.

Ask the subject, not a proxy for it: run the profiler, call the lint function, count the processes. When a claim is about a rule, ask the rule — import the module and call it rather than reimplementing its pattern in a grep.

Three numbers, not one

"Slow" is three different measurements and they lead to three different decisions.

python3 tools/sweep-selftests.py             # wall clock: what you wait for
python3 tools/sweep-selftests.py --jobs 1    # the serial shape, for a bisect

# total child CPU: what a 2-core CI runner pays, since its wall clock is ~CPU/2
import resource, subprocess, sys, time
b = resource.getrusage(resource.RUSAGE_CHILDREN); t0 = time.time()
subprocess.run([sys.executable, "tools/sweep-selftests.py"], capture_output=True)
a = resource.getrusage(resource.RUSAGE_CHILDREN)
print(time.time() - t0, (a.ru_utime - b.ru_utime) + (a.ru_stime - b.ru_stime))