avoid-warp-divergence

Skill: Avoid Warp Divergence

Purpose

Guide the agent through identifying, classifying, and restructuring warp divergence in CUDA kernels — distinguishing avoidable from unavoidable divergence, applying correct restructuring strategies, and assessing the real performance impact before spending engineering effort.

Use this when

• Profiling shows a high sm__thread_inst_executed_pipe_alu_pred_on_pct / sm__inst_executed ratio indicating significant predicated-off execution, or Nsight Compute's branch_efficiency is notably below 100%
• The kernel contains data-dependent branches where different threads in a warp take different paths based on their input data
• The kernel has irregular loop bounds or early-exit conditions that vary per thread
• Sparse or masked computation (e.g., attention masks, pruning masks) introduces branches that not all threads in a warp satisfy

Do not use this when

• The branches are on values that are uniform across the entire warp (e.g., checking a kernel argument or a loop bound that all threads compute identically): these do not cause divergence
• The divergent code is only at the boundary of the input (e.g., the last partial tile): this is unavoidable boundary divergence, and the performance impact is typically negligible for large inputs
• The kernel is memory-bandwidth-bound: eliminating divergence in a memory-bound kernel may not improve throughput because the bottleneck is not instruction throughput
• The divergence restructuring would require significant data reorganization that introduces worse memory access patterns: weigh the tradeoff explicitly

Inputs the agent should gather first

• The divergent code region: what is the branch condition? Is it a simple if/else, a loop with data-dependent bounds, or an early exit?
• What determines the branch: is it a function of thread index only (geometry-based), a function of input data values (data-dependent), or a function of a uniform warp-level value?
• Warp occupancy of each branch path: roughly what fraction of threads in a warp are expected to take each path? If 31 out of 32 threads take the same path and 1 takes the other, the impact is low. If 16 take each path, the serialization cost is highest.
• Branch body complexity: how many instructions are in the divergent region? A 2-instruction divergent branch has negligible cost even with full serialization. A 100-instruction divergent loop body is expensive.
• Hardware target: SM architecture. All current NVIDIA GPUs use the same 32-thread warp SIMT model.
• Whether the divergence is data-dependent or geometry-based: this determines which restructuring strategies apply.