cuda-kernel-optimizer

CUDA Kernel Iterative Optimizer (v2 — Roofline-Driven)

What this skill does

Given:

• a baseline kernel file (.cu for CUDA / CUTLASS, or .py for Triton),
• a reference Python file (exposes reference(**kwargs) — same contract as benchmark.py --ref),
• kernel dimension arguments (e.g. --M=4096 --N=4096 --K=4096),
• optional iteration count N (default 3), ncu_num (default 5), and branches (default 4),

the skill runs a roofline-guided, branch-and-select iterative optimization loop and produces a timestamped directory of per-iteration artifacts plus a final summary.

Key point

1. Roofline-driven axis budget: compute/memory/latency axis budgets are allocated proportionally to measured Δ gaps, with a per-axis cap of 2.
2. Branch-and-Select: each iteration generates K candidate kernels (hyperparameter/implementation variants), benchmarks all, selects champion.
3. Ablation attribution: after selecting champion, each method is individually ablated to determine its actual contribution.
4. SASS verification: cuobjdump --dump-sass confirms claimed optimizations actually appear in generated code.
5. Every iteration produces a full ncu report on the champion kernel.

Inputs the skill expects from the user

Before starting, confirm you have:

1. Baseline operator file, e.g. ./gemm.cu or ./gemm_triton.py
2. Reference file, e.g. ./ref.py (required — correctness validation depends on it)
3. Dimensions — kernel-signature scalars like --M=4096 --N=4096 --K=4096
4. Iteration count N (default 3)
5. ncu_num — how many top metrics to extract per axis (default 5)
6. branches — how many hyperparameter variants per iteration (default 4)

benchmark.py is bundled at scripts/benchmark.py; all scripts default to it automatically.

If any of these are missing, ask the user once — briefly — then proceed.

The loop at a glance

0. check_env          → env.json (GPU, nvcc, CUTLASS, ncu)
1. init run folder    → run_YYYYMMDD_HHMMSS/
2. copy baseline      → baseline/ + bench once to seed `best`
3. for i in 1..N:
     a. profile best_kernel with ncu (--set full)  → iterv{i}/best_input.ncu-rep
     b. extract top compute/mem/latency            → ncu_top.json
     c. roofline.py: compute Δ_c, Δ_m, Δ_l        → roofline.json + axis_budget
        if near_peak (all Δ < 0.15) → early stop
     d. Claude picks methods (b_axis per axis, cap=2) → analysis.md (CoT)
     e. Claude writes K branch kernels (same methods, diff hyperparams)
     f. branch_explore.py: compile + bench all K   → select champion
     g. if champion FAIL: regenerate (max 3 retries)
     h. ncu profile champion (--set full)          → iterv{i}/kernel.ncu-rep
     i. ablate.py: single-method rollback bench    → attribution.json
     j. sass_check.py: verify SASS signatures      → sass_check.json
     k. update state with attribution + SASS results
4. emit summary.md