tensormux

tensormux/kernel-skills

Instructions file

Claude Code instructions for tensormux/kernel-skills, covering claude.md, kernel-skills, repository mission, what this repository is and product scope for v1.

tensormux/kernel-skills

Skill Claude CodeCodex

Guide the agent through identifying, classifying, and restructuring warp divergence in CUDA kernels — distinguishing avoidable from unavoidable divergence, applying correct restructuring strategies, and assessing the real performance impact before spending engineering effort.

tensormux/kernel-skills

Skill Claude CodeCodex

Guide the agent through selecting the correct and efficient thread block dimensions and grid dimensions for a CUDA kernel, covering occupancy analysis, register and shared memory constraints, tail effects, persistent kernels, and when to use cudaOccupancyMaxActiveBlocksPerMultiprocessor as a decision tool.

tensormux/kernel-skills

Skill Claude CodeCodex

Guide the agent through a systematic process of isolating, reproducing, and diagnosing correctness errors in CUDA kernels — covering indexing bugs, layout mismatches, synchronization races, reduction errors, numerical drift, and out-of-bounds memory access.

tensormux/kernel-skills

Skill Claude CodeCodex

Guide the agent through diagnosing and restructuring CUDA global memory access patterns to maximize effective memory bandwidth, covering coalescing requirements, vectorized loads, AoS vs SoA layouts, shared memory staging for non-coalesced patterns, and L2 cache behavior.

tensormux/kernel-skills

Skill Claude CodeCodex

Guide the agent through designing and tuning shared memory tiling strategies for CUDA kernels, covering bank conflict analysis and elimination, tile shape selection, double buffering with async copy, occupancy tradeoffs from shared memory allocation, and the decision of when smem tiling is worth the complexity.

tensormux/kernel-skills

Skill Claude CodeCodex

Guide the agent through designing and implementing a correct, performance-aware CUDA GEMM kernel (C = alpha A B + beta C) for a specific problem configuration, including decisions about tiling strategy, memory hierarchy usage, tensor core eligibility, and when to defer to cuBLAS or CUTLASS instead.

tensormux/kernel-skills

Skill Claude CodeCodex

Guide the agent through designing and implementing a correct, efficient CUDA LayerNorm (and RMSNorm) kernel, covering mean/variance computation strategies, Welford online accumulation, epsilon placement, affine transform application, backward pass structure, and decomposition for non-power-of-two hidden dimensions.

tensormux/kernel-skills

Skill Claude CodeCodex

Guide the agent through designing and implementing a correct, efficient CUDA reduction kernel for a given operator (sum, max, min, or custom binary associative op), covering warp-level primitives, block-level reduction, multi-block strategies, and when to use CUB instead.

tensormux/kernel-skills

Skill Claude CodeCodex

Guide the agent through designing and implementing a correct, numerically stable CUDA softmax kernel, covering online (single-pass) computation, row-parallel decomposition, warp-level reductions, fp16/bf16 precision pitfalls, masked softmax variants, and when to fuse with attention versus implementing standalone.

tensormux/kernel-skills

Skill Claude CodeCodex

Guide the agent through choosing and tuning kernels for the prefill phase versus the decode phase of LLM inference. The two phases have fundamentally different arithmetic intensity, occupy different sides of the roofline, and respond to different optimizations. Continuous batching and speculative decoding shift the…

tensormux/kernel-skills

Skill Claude CodeCodex

Guide the agent through planning how a custom CUDA kernel will be wrapped as a TensorRT plugin so it can be invoked from inside a TensorRT engine — covering API choice (IPluginV3 vs IPluginV2DynamicExt), the plugin lifecycle, dynamic shape handling, serialization, mixed precision (FP16/INT8/FP8), workspace management…

tensormux/kernel-skills

Skill Claude CodeCodex

Guide the agent through implementing a Triton kernel that unpacks and dequantizes a quantized weight tensor (int4 or int8) into fp16 or bf16. This is the standalone building block underneath W4A16 / W8A16 schemes (AWQ, GPTQ, SqueezeLLM, bitsandbytes NF4, int8 per-channel). Covers bit-unpacking, per-group scale/zero…

tensormux/kernel-skills

Skill Claude CodeCodex

Guide the agent through implementing a single Triton kernel that computes y = rmsnorm(x + residual) while also writing back x + residual for the next transformer block's residual stream. This fusion is the dominant pattern in LLaMA, Mistral, Qwen, and similar decoder blocks: every attention sub-block and every MLP…

tensormux/kernel-skills

Skill Claude CodeCodex

Guide the agent through implementing a Triton kernel that writes newly computed K and V tensors into a pre-allocated KV cache during LLM inference. This covers two cache layouts (contiguous and paged / vLLM-style PagedAttention), unified prefill and decode handling via a slotmapping tensor, GQA/MQA where the cache…

tensormux/kernel-skills

Skill Claude CodeCodex

Guide the agent through implementing a correct, numerically stable RMSNorm kernel in Triton: y = x rsqrt(mean(x², axis=-1) + eps) weight. RMSNorm is the dominant normalization in modern decoder-only LLMs (LLaMA, Mistral, Qwen, Gemma, DeepSeek). This skill covers one-pass sum-of-squares with fp32 accumulation, the…

tensormux/kernel-skills

Skill Claude CodeCodex

Guide the agent through implementing a correct Triton kernel that applies Rotary Position Embeddings (RoPE) to query and key tensors before attention. This covers the two incompatible layout conventions (GPT-NeoX/HuggingFace-LLaMA vs GPT-J/original-paper), pre-computed cos/sin table consumption, per-token position…

tensormux/kernel-skills

Skill Claude CodeCodex

Guide the agent through implementing a Triton kernel for LLM decode-time token sampling: take a [batch, vocab] logits tensor, apply per-request temperature, top-k, and top-p (nucleus) filtering, renormalize, and draw one token per request. This is the last hot kernel on every decode step — it runs once per generated…

tensormux/kernel-skills

Skill Claude CodeCodex

Guide the agent through implementing a correct, numerically stable Triton kernel that computes y = silu(a) b, the elementwise activation step inside SwiGLU MLPs used by LLaMA, Mistral, Qwen, Gemma, and similar modern LLMs. The full MLP is downproj( silu(gateproj(x)) upproj(x) ); this skill covers the fused activation…

tensormux/kernel-skills

Skill Claude CodeCodex

Guide the agent through planning the integration of a custom CUDA or Triton kernel into the vLLM inference engine before any integration code is written — covering where the op plugs into the engine, paged KV cache and continuous batching compatibility, CUDA graph capture constraints, tensor parallelism implications…

tensormux/kernel-skills

Skill Claude CodeCodex

Guide the agent through selecting tile sizes and work partitioning strategies for a CUDA or Triton kernel, based on shared memory budget, register pressure, occupancy targets, problem shape, and access pattern.

tensormux/kernel-skills

Skill Claude CodeCodex

Guide the agent through deciding whether to fuse multiple elementwise operations into a single kernel pass, and if so, how to implement the fusion correctly and efficiently in CUDA or Triton.

tensormux/kernel-skills

Skill Claude CodeCodex

Guide the agent through correctly handling partial tiles — cases where a problem dimension does not evenly divide the tile size — in CUDA and Triton kernels, without introducing out-of-bounds accesses, incorrect output values, or silent data corruption.

tensormux/kernel-skills

Skill Claude CodeCodex

Guide the agent through constructing a systematic, coverage-complete test plan for a compute kernel, covering correctness, numerical precision, boundary conditions, layout variations, and performance regression.