guoqingbao/xinfer

Blazing-fast LLM inference in pure Rust. No PyTorch and Python runtime.

314Stars on the repository
3Mods indexed here, across every type
3d agoLast push, which is what freshness is scored on
MITLicence, which decides whether bodies are shown

check-model

01

guoqingbao/xinfer

Skill Cursor

Check model compatibility with xinfer before loading. Validates config.json, weight tensor shapes and naming, quantization format correctness, and multi-rank (tensor-parallel) divisibility. Use when the user asks to check, validate, audit, or verify a model will load correctly — from a HuggingFace URL/config, local…

not rated 314 +2 3d ago A 76 tokens original MIT

test-model

02

guoqingbao/xinfer

Skill Cursor

Test LLM models served by xinfer for correctness, output quality, and performance. Use when the user asks to test, benchmark, validate, or verify models — either from a local folder path or HuggingFace model IDs. Supports all xinfer-compatible formats: BF16, FP8, MXFP4, NVFP4, GGUF, GPTQ, AWQ, ISQ, Dense, MoE, and…

not rated 314 +2 3d ago A 93 tokens original MIT