write-triton-dequant-kernel

Skill: Write a Triton Dequant Kernel (int4 / int8 → fp16 / bf16)

Purpose

Guide the agent through implementing a Triton kernel that unpacks and dequantizes a quantized weight tensor (int4 or int8) into fp16 or bf16. This is the standalone building block underneath W4A16 / W8A16 schemes (AWQ, GPTQ, SqueezeLLM, bitsandbytes NF4, int8 per-channel). Covers bit-unpacking, per-group scale/zero arithmetic, NF4 codebook lookup, and — critically — when not to write a standalone dequant kernel because the operation should be fused into the matmul instead.

---

Use this when

• You need to materialize a fp16/bf16 weight tile from a packed int4/int8 representation for a consumer that does not have a fused-dequant matmul (custom op, MoE expert hot-swap, LoRA merge into a quantized base).
• You need a baseline reference to validate a fused dequant-GEMM (Marlin, AWQ-fused-GEMM, ExLlama, GPTQ kernels) by comparing intermediate fp16 weights.
• You are debugging a quantized model's accuracy and need to inspect the dequantized weights for a specific scheme (AWQ packing order, GPTQ act-order permutation).
• You are implementing a one-off or research scheme and a fused kernel does not yet exist.
• You must dequantize on the fly because the quantized weight is being modified at runtime (online calibration, dynamic LoRA composition).

---

Do not use this when

• You are running a standard W4A16 or W8A16 linear layer in production. Standalone dequant materializes the full fp16 weight to HBM and reads it back for the GEMM, doubling weight memory traffic and defeating most of the benefit of weight quantization. Use a fused dequant-GEMM instead (Marlin / AWQ kernels / GPTQ kernels / ExLlama / torch.ops.aten._weight_int4pack_mm).
• The downstream GEMM is bandwidth-bound on the weight (typical for batch size 1–32 LLM decode). A fused kernel is strictly better — standalone dequant turns one HBM read of int4 into one read of int4 + one write of fp16 + one read of fp16.
• You only need activation dequantization. This skill is about weight dequant; activation dequant lives in the GEMM epilogue.
• The scheme is symmetric int8 per-tensor and the framework already has torch.dequantize or equivalent.
• You want to "speed up" inference by replacing a fused GEMM with standalone dequant + cuBLAS fp16 GEMM. Almost always slower.