FlashInfer is a library and kernel generator that supplies GPU operations used to run large language model inference, including attention, matrix multiplication, and mixture-of-experts computations. It helps engineers build and optimize LLM serving systems across supported GPU hardware and backend implementations. Its catalogue add-ons provide skills and instructions for working with FlashInfer.
Borrowing it
Nothing to install: this file belongs to flashinfer-ai/flashinfer. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.
curl -O https://raw.githubusercontent.com/flashinfer-ai/flashinfer/main/.claude/skills/debug-cuda-crash/SKILL.mdgit clone --depth 1 https://github.com/flashinfer-ai/flashinferWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/flashinfer-ai/flashinfer/debug-cuda-crash)<a href="https://agentmods.dev/skills/flashinfer-ai/flashinfer/debug-cuda-crash"><img src="https://agentmods.dev/badge/skills/flashinfer-ai/flashinfer/debug-cuda-crash.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00014 | $0.03612 |
| Opus 5 | $0.00007 | $0.01806 |
| Sonnet 5 | $0.00003 | $0.00722 |
| Haiku 4.5 | $0.00001 | $0.00361 |
Grade A, and why
debug-cuda-crash scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 7d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 575 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Tutorial: Debugging CUDA Crashes with API Logging
This tutorial shows you how to debug CUDA crashes and errors in FlashInfer using the @flashinfer_api logging decorator.
Goal
When your code crashes with CUDA errors (illegal memory access, out-of-bounds, NaN/Inf), use API logging to:
- Capture input tensors BEFORE the crash occurs
- Understand what data caused the problem
- Track tensor shapes, dtypes, and values through your pipeline
- Detect numerical issues (NaN, Inf, wrong shapes)
Why Use API Logging?
Problem: CUDA errors often crash the program, leaving no debugging information.
Solution: FlashInfer's @flashinfer_api decorator logs inputs BEFORE execution, so you can see what caused the crash even after the program terminates.
Step 1: Enable API Logging
Basic Logging (Function Names Only)
export FLASHINFER_LOGLEVEL=1 # Log function names
export FLASHINFER_LOGDEST=stdout # Log to console
python my_script.py
Output:
[2025-12-18 10:30:45] FlashInfer API Call: batch_decode_with_padded_kv_cache
Detailed Logging (Inputs/Outputs with Metadata)
export FLASHINFER_LOGLEVEL=3 # Log inputs/outputs with metadata
export FLASHINFER_LOGDEST=debug.log # Save to file
python my_script.py
Output in debug.log:
================================================================================
[2025-12-18 10:30:45] FlashInfer API Logging - System Information
================================================================================
FlashInfer version: 0.6.0
CUDA toolkit version: 12.1
GPU 0: NVIDIA H100 PCIe
Compute capability: 9.0 (SM90)
PyTorch version: 2.1.0
================================================================================
================================================================================
[2025-12-18 10:30:46] FlashInfer API Call: batch_decode_with_padded_kv_cache
--------------------------------------------------------------------------------
Positional input arguments:
arg[0]:
Tensor(
shape=(32, 8, 128)
dtype=torch.bfloat16
device=cuda:0
requires_grad=False
is_contiguous=True
)
Keyword input arguments:
kv_cache=
Tensor(
shape=(1024, 2, 8, 128)
dtype=torch.bfloat16
device=cuda:0
requires_grad=False
is_contiguous=True
)
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 7d ago First seen · 575 lines · 14 tokens per session scan A d0b39248d13e
debug-cuda-crash is a skill published in the GitHub repository flashinfer-ai/flashinfer (6,340 stars, last pushed yesterday), licensed Apache-2.0. It adds 14 tokens to every session and 3,612 once invoked, about $0.0001 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
perf-analyze
Launch and operate this repo's perf-analyze workflow, which DIAGNOSES a TensorRT-LLM serving deployment without applying changes — benchmark at one concurrency or a Pareto curve of them (tok/s/user vs tok/s/gpu), analytical SOL projection on by default (via the internal-perf-sol-analysis skill), nsys + torch-profiler…
cuda-kernel-optimizer
Use when optimizing or diagnosing GPU workload performance with CUDA, CUTLASS, Triton, PyTorch, vLLM or TensorRT-LLM; interpreting NCU, Nsys or PyTorch Profiler evidence; or preparing the workload and correctness checks for such optimization. General coding, skill maintenance and conceptual GPU questions do not…
gpu-server-setup
Prepare a Linux server with NVIDIA GPUs for neural-network and LLM workloads, or diagnose one that misbehaves. Use when the user asks to set up or prepare a GPU server, install NVIDIA drivers or CUDA on Debian/Ubuntu, wire Docker to GPUs (NVIDIA Container Toolkit), deploy vLLM / Infinity / OpenWebUI / Ollama /…
sglang-diffusion-benchmark-profile
Use when benchmarking denoise latency or profiling a diffusion bottleneck in SGLang.
test-amplification
Methodical bug-driven test amplification for pygraphistry features. Use when hardening a feature area via user-workflow exploration, 5-Whys retrospectives, bug-taxonomy derivation, concrete test planning, implementation, and safety-gated validation.
neuron-debugger
Debug and monitor Neuron AI applications with Inspector APM, event observability, logging, and performance analysis. Use this skill whenever the user mentions debugging, monitoring, observability, performance analysis, tracing, Inspector, or needs to understand why an agent is behaving a certain way. Also trigger for…