flashinfer-ai/flashinfer

FlashInfer is a library and kernel generator that supplies GPU operations used to run large language model inference, including attention, matrix multiplication, and mixture-of-experts computations. It helps engineers build and optimize LLM serving systems across supported GPU hardware and backend implementations. Its catalogue add-ons provide skills and instructions for working with FlashInfer.