debug-cuda-crash

Tutorial: Debugging CUDA Crashes with Kernel API Logging

This tutorial shows you how to debug CUDA crashes and errors in SGLang using the @debug_kernel_api logging decorator.

Goal

When your code crashes with CUDA errors such as illegal memory access, device-side assert, out-of-bounds, or NaN/Inf, use kernel API logging to:

• Capture input tensors BEFORE the crash occurs
• Understand what data caused the problem
• Track tensor shapes, dtypes, and values through the call boundary that triggered the crash
• Detect numerical issues such as NaN, Inf, or obviously wrong shapes

Why Use Kernel API Logging?

Problem: CUDA errors often crash the program before normal debugging output is flushed.

Solution: SGLang's @debug_kernel_api decorator logs inputs before execution, so you can still see what caused the crash even after the program aborts.

What Is Covered?

The current logging coverage focuses on the highest-value kernel boundaries in SGLang:

• Custom ops registered through register_custom_op(...)
• External custom ops registered through register_custom_op_from_extern(...)
• LLM attention, linear, quantization, and multi-platform wrapper entry points
• Diffusion attention impl, linear, rotary, and custom-op wrapper entry points
• Selected direct torch.ops.sglang.* hotspots and model-specific bypasses

This means the logging is useful for both LLM and diffusion kernel debugging, but it does not automatically cover every pure PyTorch call in the repository.

Step 1: Enable Kernel API Logging

Basic Logging (Function Names Only)

export SGLANG_KERNEL_API_LOGLEVEL=1
export SGLANG_KERNEL_API_LOGDEST=stdout

python my_script.py

Output:

================================================================================
[2026-03-19 00:47:06] SGLang Kernel API Call: RMSNorm.forward
================================================================================
[2026-03-19 00:47:06] SGLang Kernel API Call: sglang.quant_method.UnquantizedLinearMethod.apply
================================================================================
[2026-03-19 00:47:06] SGLang Kernel API Call: sglang.custom_op.fused_inplace_qknorm