aiter-reflection

Aiter Reflection

Overview

Optimize AMD MI300 GPU kernels for correctness and performance using the aiter workflow, then record each iteration to the kernel experiment database.

Workflow

1) Locate targets and understand tests

• Use the provided context to identify target kernel files, kernels, and their op tests.
• Run the op tests once to understand output format and verify correctness expectations. (Attention: Stucked background op test processes and lock files under jit folder may cause the op tests running failed; Op tests require JIT compiling, please be prepared to wait for a long time)

2) Build a benchmark shell script

• Come up with a new name for this iteration and create a folder logs/. Put the shell script under this folder
• Reuse the existing op_test python script
• Covers common shapes: 128, 256, 512, 1024, 2048, 4096 if applies
• Repeats each op test multiple times and reports the correctness and the average time consuming.
  • Use at least 100 iterations per configuration for reliable results
  • Include 10-20 warmup iterations to handle JIT compilation overhead
  • Add torch.cuda.synchronize() after each kernel call
  • Use fixed random seed for reproducibility
  • Use high-precision timing (time.perf_counter())
• Implements a robust timeout to avoid hangs.
• Outputs structured timing per shape.

3) Establish a baseline

• Before testing: Check for background GPU processes that may interfere
  • Use rocm-smi or ps aux | grep python to identify GPU tasks
  • Stop any unrelated GPU workloads
• Clear JIT compilation cache to ensure clean state
• Run the benchmark script using the .venv Python environment
• Save results under logs/ folder with timestamp

4) Iterate on kernel optimization (one iteration)

• Read the kernel source, identify bottlenecks, and call rocprof-compute at least once to deepen bottleneck analysis.
• Use kernel-exp-history to review related optimization history and extract ideas.
• Modify the kernel file to improve performance for multiple shapes allowed.
• Save the changes: (git diff > logs//iter_diff.patch)
• Reinstall aiter and clear cache:
  • python -m pip install -e . --no-build-isolation --no-deps --force-reinstall
  • rm -f aiter/jit/*.so && rm -rf aiter/jit/build ~/.aiter
• Re-run the benchmark to measure the new performance.
• If results seem suspicious (unexpected regressions):
  • Verify no background processes are running
  • Re-test baseline with same methodology
  • Check if JIT compilation overhead affected measurements