flashinfer: Skill for Claude Code

Tutorial: Benchmarking FlashInfer Kernels

This tutorial shows you how to accurately benchmark FlashInfer kernels.

Goal

Measure the performance of FlashInfer kernels:

• Get accurate GPU kernel execution time
• Compare multiple backends (FlashAttention2/3, cuDNN, CUTLASS, TensorRT-LLM)
• Generate reproducible benchmark results
• Save results to CSV for analysis

Timing Methods

FlashInfer supports two timing methods:

1. CUPTI (Preferred): Hardware-level profiling for most accurate GPU kernel time

   • Measures pure GPU compute time without host-device overhead
   • Requires cupti-python >= 13.0.0 (CUDA 13+)

2. CUDA Events (Fallback): Standard CUDA event timing

   • Automatically used if CUPTI is not available
   • Good accuracy, slight overhead from host synchronization

The framework automatically uses CUPTI if available, otherwise falls back to CUDA events.

Autotuner timing (separate from the benchmark framework above). The AutoTuner's internal per-tactic timing has its own selector, FLASHINFER_AUTOTUNE_TIMER: globaltimer forces the GPU %globaltimer register, cuda_event forces cudaEvent, and unset/auto uses %globaltimer only when Confidential Computing (CC) is detected. Under CC cudaEventElapsedTime is unreliable (can go negative), which would corrupt tactic ranking — the globaltimer path avoids that. CC auto-detection can be overridden with FLASHINFER_CONFIDENTIAL_COMPUTE=0/1. (Full env-var reference in CLAUDE.md.)

Installation

Install CUPTI (Recommended)

For the most accurate benchmarking:

pip install -U cupti-python

Requirements: CUDA 13+ (CUPTI version 13+)

Without CUPTI

If you don't install CUPTI, the framework will:

• Print a warning: CUPTI is not installed. Falling back to CUDA events.
• Automatically use CUDA events for timing
• Still provide good benchmark results

Method 1: Using flashinfer_benchmark.py (Recommended)

Step 1: Choose Your Test Routine