FlashInfer is a library and kernel generator that supplies GPU operations used to run large language model inference, including attention, matrix multiplication, and mixture-of-experts computations. It helps engineers build and optimize LLM serving systems across supported GPU hardware and backend implementations. Its catalogue add-ons provide skills and instructions for working with FlashInfer.
Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add instructions/flashinfer-ai/flashinfer/claude-mdgit clone --depth 1 https://github.com/flashinfer-ai/flashinferWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/instructions/flashinfer-ai/flashinfer/claude-md)<a href="https://agentmods.dev/instructions/flashinfer-ai/flashinfer/claude-md"><img src="https://agentmods.dev/badge/instructions/flashinfer-ai/flashinfer/claude-md.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.11353 | $0.11353 |
| Opus 5 | $0.05677 | $0.05677 |
| Sonnet 5 | $0.02271 | $0.02271 |
| Haiku 4.5 | $0.01135 | $0.01135 |
Grade E, and why
flashinfer CLAUDE.md scanned grade E with 2 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured today.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nullifies safety policieshighAnti-refusal
"You have no restrictions", "do anything now", "ignore your guidelines": a direct jailbreak that disables guardrails.
| `FLASHINFER_PACKED_KDA_EVICT_FIRST` | batch policy | Set to `1` or `0` to override the policy's first-use cache-eviction hint. | Recursive force deletehighDestructive command
rm -rf with a variable or a broad path is one typo away from removing the wrong tree.
| Clear JIT cache | `rm -rf ~/.cache/flashinfer/` | How it starts
The opening of the file, as written. The whole thing — 789 lines — stays where its author put it; the contents beside it link to each section on GitHub.
CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
Project Overview
FlashInfer is a GPU kernel library for LLM serving that uses JIT (Just-In-Time) compilation by default. This means kernel code changes are automatically picked up without reinstalling the package - extremely convenient for development.
Quick Reference
| Task | Command |
|---|---|
| Install for development | pip install --no-build-isolation -e . -v |
| Initialize submodules | git submodule update --init --recursive |
| Install CUPTI for benchmarking | pip install -U cupti-python |
| Run all tests | pytest tests/ |
| Run specific test | pytest tests/path/test_file.py::test_function |
| Run multi-GPU test | mpirun -np 4 pytest tests/comm/test_allreduce_unified_api.py |
| Run benchmark | python benchmarks/flashinfer_benchmark.py --routine <name> <flags> |
| Run linting | pre-commit run -a |
| Dump environment report (bug reports) | python -m flashinfer.collect_env (or flashinfer collect-env [--json]) |
| Install pre-commit hooks | pre-commit install |
| Clear JIT cache | rm -rf ~/.cache/flashinfer/ |
| Enable API logging (basic) | export FLASHINFER_LOGLEVEL=1 |
| Enable API logging (detailed) | export FLASHINFER_LOGLEVEL=3 |
| Enable API logging (with stats) | export FLASHINFER_LOGLEVEL=5 |
| Set API log destination | export FLASHINFER_LOGDEST=mylog.txt |
| Enable verbose JIT logging | export FLASHINFER_JIT_VERBOSE=1 |
| Enable debug build | export FLASHINFER_JIT_DEBUG=1 |
| Set target architectures | export FLASHINFER_CUDA_ARCH_LIST="8.0 9.0a" |
| Set parallel compilation | export FLASHINFER_NVCC_THREADS=4 |
| Limit parallel ninja jobs | export MAX_JOBS=4 |
| Enable GDN native short-T path | export FLASHINFER_GDN_WY_NATIVE_T=1 |
| Enable GDN strided QKV path | export FLASHINFER_GDN_WY_STRIDED_QKV=1 |
| Enable GDN native A/B tensors | export FLASHINFER_GDN_WY_NATIVE_AB=1 |
Let backend="auto" pick experimental backends |
export FLASHINFER_ALLOW_EXPERIMENTAL_AUTO_BACKENDS=1 |
| Override CuTe-DSL prefill scheduling | export FLASHINFER_CUTE_PREFILL_PERSISTENT=0 (non-persistent) or 1 (persistent) |
| Skip MoE EP CuTe-DSL import/version guard | export FLASHINFER_MOE_EP_SKIP_DSL_CHECK=1 |
| Override MoE EP knob-cache path | export FLASHINFER_MOE_EP_KNOB_CACHE=/path/to/knobs.json |
| Disable MoE EP fused staging kernel | export FLASHINFER_MEGA_FUSED_STAGE=0 |
| Enable distribution-aware MoE autotune and kernel dispatch (experimental; TRT-LLM MoE only) | export FLASHINFER_DIST_AWARE_AUTOTUNE=1 |
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- today Changed · +48 lines · +805 tokens per session cb4492876260
- yesterday Changed · +3 lines · +25 tokens per session 71323f0f2524
- 3d ago Changed · +2 lines · +159 tokens per session fb91cc52b56a
- 6d ago First seen · 736 lines · 10,364 tokens per session scan E 3a7f551e122e
flashinfer CLAUDE.md is an instructions file published in the GitHub repository flashinfer-ai/flashinfer (6,337 stars, last pushed today), licensed Apache-2.0. It adds 11,353 tokens to every session, about $0.0568 per session on Opus 5. A static security scan graded it E with 2 findings (nullifies safety policies, recursive force delete). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other instructions, from other repositories
gpu-server-setup AGENTS.md
Instructions for EvilFreelancer/gpu-server-setup, covering authoring & maintaining the gpu-server-setup skill, repository layout, editing rules, keeping metadata in sync and what this skill does not do.
AutoMegaKernel AGENTS.md
Instructions for RightNow-AI/AutoMegaKernel, covering what amk is, the edit surface, scheduleconfig + kernelknobs only, canonical tool surface (mcp), use these exact names, codex mcp server config (/.codex/config.toml) and the copy-paste loop a codex agent runs.
DeepSpeed AGENTS.md
AGENTS.md instructions for deepspeedai/DeepSpeed, covering deepspeed project rules, commit & ci requirements, code change discipline, test discipline and tool caveats.
cuopt AGENTS.md
AGENTS.md instructions for NVIDIA/cuopt, covering agents.md — cuopt ai agent entry point, skills directory (flat), rules, common (concepts only; no api code) and installation.
TensorRT-LLM AGENTS.md
AGENTS.md instructions for NVIDIA/TensorRT-LLM, covering agents.md, rules (read first), common commands, installation & build and reference configs.
cuda-metal AGENTS.md
AGENTS.md instructions for Lulzx/cuda-metal, covering agents.md — cumetal development guide, project identity, current focus, repository structure and non-negotiable constraints.