cuda-kernel-optimizer

CUDA Kernel Iterative Optimizer (v3)

What this skill does

Given:

• a baseline kernel file (.cu for CUDA / CUTLASS, or .py for Triton),
• a reference Python file (exposes reference(**kwargs) — same contract as benchmark.py --ref),
• optional atol/rtol and workload_model(**dims) -> {flops, bytes_min} in the reference,
• kernel dimension arguments (e.g. --M=4096 --N=4096 --K=4096),
• optional iteration count N (default 3), ncu_num (default 5), and branches (default 4),
• optional --compile-jobs auto|N (default auto) and --numerics-mode reference|strict|approximate (default reference),

the skill runs a roofline-guided, branch-and-select iterative optimization loop and produces a timestamped directory of per-iteration artifacts plus a final summary.

Key point

1. Evidence-driven axis budget: use a real workload roofline when ref.py exposes workload_model; otherwise label the result as a bottleneck-gap heuristic and do not claim near_peak.
2. Branch-and-Select: each iteration generates K candidate kernels (hyperparameter/implementation variants), benchmarks all, selects champion.
3. Ablation attribution: after selecting champion, each method is individually ablated to determine its actual contribution.
4. SASS verification: cuobjdump --dump-sass confirms claimed optimizations actually appear in generated code.
5. Every iteration produces an adaptive ncu report on the champion kernel; only CPU compilation is parallel.
6. No evidence, no generation: the requested iteration count is a maximum. Never write candidates until the hardware, correctness, timing, and NCU gates pass.

Inputs the skill expects from the user

Before starting, confirm you have:

1. Baseline operator file, e.g. ./gemm.cu or ./gemm_triton.py
2. Reference file, e.g. ./ref.py (required — correctness validation depends on it)
3. Dimensions — kernel-signature scalars like --M=4096 --N=4096 --K=4096
4. Iteration count N (default 3)
5. ncu_num — how many top metrics to extract per axis (default 5)
6. branches — how many hyperparameter variants per iteration (default 4)