SGLang is a framework for running inference for large language models and multimodal models, meaning it processes inputs to produce model outputs such as text or other media. It is used to serve and accelerate open AI models and related workloads.
Borrowing it
Nothing to install: this file belongs to sgl-project/sglang. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.
curl -O https://raw.githubusercontent.com/sgl-project/sglang/main/.agents/skills/kernel-organization/SKILL.mdgit clone --depth 1 https://github.com/sgl-project/sglangWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/sgl-project/sglang/kernel-organization)<a href="https://agentmods.dev/skills/sgl-project/sglang/kernel-organization"><img src="https://agentmods.dev/badge/skills/sgl-project/sglang/kernel-organization/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/sgl-project/sglang/kernel-organization"><img src="https://agentmods.dev/badge/skills/sgl-project/sglang/kernel-organization.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00062 | $0.01667 |
| Opus 5.5 | $0.00025 | $0.00667 |
| Sonnet 5.5 | $0.00012 | $0.00333 |
| Haiku 4.5 | $0.00006 | $0.00167 |
Grade A, and why
kernel-organization scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 4d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 115 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Kernel organization
Read python/sglang/kernels/README.md and test/README.md before choosing a
location. RFC #29630
establishes the namespace; subsequent migrations #32148 and #40922 clarify
logical grouping and remove the old _jit_ filename prefix.
Choose the owner
- Put callable SGLang kernel APIs under
sglang.kernels.ops.<group>. Runtime and integration tests import from that namespace, including its submodules. Thesgl_kernelwheel retains its own public API and packaging tests. - Group by computation, not model or GPU: GEMM and GEMV in
gemm, expert routing inmoe, attention index selection inattention, sampling insampling, normalization inlayernorm. - Classify a fused operator by its complete contract; do not split a fused
kernel into separate launches just to separate its parts. Split unrelated
public entry points that happen to share a source file. Decided rules:
- Any cross-rank collective or peer-memory access (all-reduce, all-gather,
reduce-scatter, a2a, multicast/NVLS) puts the op in
communication, whatever compute it fuses. - Producer + output quantization stays with the producer (norm + quant in
layernorm, activation + quant inactivation, GEMM with a quant epilogue ingemm, MoE dispatch/permute + quant inmoe).quantizationholds only pure format conversion: quant, dequant, scale-layout conversion, repack. - RoPE stays in
attention(standalone RoPE, position ids, QK-norm + RoPE). kvcacheowns ops that read/write KV-cache contents;attentionowns the index/shape/schedule tables (page tables,cu_seqlens,kv_indices, split metadata) and index selection. A fused compute + store stays with its compute group.
- Any cross-rank collective or peer-memory access (all-reduce, all-gather,
reduce-scatter, a2a, multicast/NVLS) puts the op in
- Model-specific files/subpackages and tuning data are allowed inside a
logical group. Do not add a model bundle or an implementation file directly
under
ops/. Propose a new logical group only for a distinct responsibility, and updateops._GROUPS, the README group list, and tests together (test_kernels_namespace.pychecks the README against_GROUPSand the README layout against the directories underkernels/). - Agent-generated implementations live in
kernels/kda_kernels/(Kernel Design Agents,KernelBackend.KDA) andkernels/cake_kernels/(Cake adapters over FlashInfer). They are backends, not groups: register their public ops in the logical group with aKernelSpecwhose target points into them, and do not reorganize those directories. - External kernel libraries that load as SGLang plugins and replace a model's
step (sgl-nvidia-mega behind
--enable-nvidia-megakernel, sgl-amd-mono behind--enable-amd-monokernel) are split at the kernel/runtime boundary. Their contracts (spec, registration, capability declarations, thin kernel wrappers) live inkernels/plugin_kernels/<hardware>/<model>/(KernelBackend.PLUGIN) and import nothing fromsrt. SGLang's integration (config checks, weight observation, CUDA graph coverage, forward dispatch, the runtime binding) lives insrt/kernel_plugins/<hardware>/binding.py. An op exposed from the contract side registers in its logical group with aKernelSpecthat points into it. - Keep shared build/runtime infrastructure (including the Helion runtime
helper) in
kernels/jit; operator wrappers call it from their logical group. Keep process groups, communicator state, workspace classes, model dispatch, and buffer ownership insrt(srt/layers/communication/for the K3 and Muse Spark adapters); kernels must not import srt runtime modules to inspect model objects, so pass flags in. Adapters there need not pretend to be generic interfaces.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 4d ago Changed · +9 lines a31729dd2317
- 8d ago Changed · +23 lines caf5a0c35d41
- 1mo ago First seen · 83 lines · 62 tokens per session scan A 04af1fc4bc45
kernel-organization is a skill published in the GitHub repository sgl-project/sglang (38,324 stars, last pushed today), licensed Apache-2.0. It adds 62 tokens to every session and 1,667 once invoked, about $0.0002 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-27.
Other skills, from other repositories
groq-inference
Ultra-fast LLM inference on custom LPU hardware. OpenAI-compatible API at api.groq.com. Lowest latency in the industry (500-1000+ tok/s). Supports chat completions, vision, audio (Whisper STT + TTS), tool calling, JSON mode, and streaming. Free tier available. Inference only — no training.
perf-optimize
Launch and operate this repo's perf-optimize workflow, which iteratively APPLIES TensorRT-LLM serving optimizations — baseline benchmark at one concurrency or a Pareto curve of them (tok/s/user vs tok/s/gpu), analytical SOL projection on by default (via the internal-perf-sol-analysis skill) sizing the headroom the…
perf-analyze
Launch and operate this repo's perf-analyze workflow, which DIAGNOSES a TensorRT-LLM serving deployment without applying changes — benchmark at one concurrency or a Pareto curve of them (tok/s/user vs tok/s/gpu), analytical SOL projection on by default (via the internal-perf-sol-analysis skill), nsys + ncu per-kernel…
fireworks-ai-inference
Fast inference and fine-tuning platform with serverless and on-demand GPU deployments. OpenAI-compatible API for chat completions, embeddings, function calling, vision, and structured output. Supports SFT, DPO, and RL fine-tuning. SOC2 + HIPAA compliant.
vllm
Use when self-hosting an OpenAI-compatible LLM endpoint with vLLM 0.32 — vllm serve, offline LLM.generate / llm.chat, PagedAttention with continuous batching and prefix caching, --tensor-parallel-size / --pipeline-parallel-size, AWQ/GPTQ/FP8 quantization, or wiring LangChain/LlamaIndex clients at…
llm-kv-cache-paged-attention
Use this skill when the user says 'KV cache', 'PagedAttention', 'vLLM memory', 'gpumemoryutilization', 'prefix caching', 'max concurrent sequences', 'KV cache size', 'context length OOM'. Explains KV cache memory math and how paged allocation, prefix caching, KV quantization, and attention variants (MQA/GQA/MLA)…