NVIDIA/TensorRT-LLM

NVIDIA/TensorRT-LLM

Skill Claude CodeCodex ✓ vendor

Identify and eliminate host-device synchronizations in PyTorch code. Detects sync points (.item(), .cpu(), boolean indexing, torch.tensor on CUDA), classifies false vs true dependencies, provides sync-free alternatives. Triggers: sync-free, synchronization, .item(), .cpu(), host-device sync, eliminate syncs, CPU…

NVIDIA/TensorRT-LLM

Skill Claude CodeCodex ✓ vendor

Code instrumentation for timing workloads. Two scenarios: (1) Training loop — inject manual timing to report per-iteration latency, throughput (samples/sec), and data load time. (2) Standalone kernel/op — write CUDA event timing code with warmup, per-iteration statistics, and anti-pattern avoidance. Also covers NVTX…

NVIDIA/TensorRT-LLM

Skill Claude CodeCodex ✓ vendor

Run TensorRT-LLM test cases, benchmarks, evaluations, or custom scripts by checking the environment (local GPU or Slurm), selecting the appropriate Docker image, and executing either locally or via Slurm job submission. Accepts pre-built command strings — command construction for trtllm-bench, trtllm-eval, and…

NVIDIA/TensorRT-LLM

Skill Claude CodeCodex ✓ vendor

Best practices for contributing code to TensorRT-LLM. Covers the official contribution process (issue tracking, fork workflow, DCO signing), coding guidelines, implementation workflow, common mistakes, testing strategy, commit hygiene, and review readiness. Incorporates rules from CONTRIBUTING.md and…

NVIDIA/TensorRT-LLM

Skill Claude CodeCodex ✓ vendor

Systematic approach to exploring the TensorRT-LLM codebase before implementing new features or optimizations. Teaches how to discover existing infrastructure, trace code paths, and avoid reimplementing what already exists. Derived from real mistakes where 250 lines of code were written and deleted because existing…

NVIDIA/TensorRT-LLM

Skill Claude CodeCodex ✓ vendor

Upgrade flashinfer-python version in TensorRT-LLM. Fetches the latest releases from GitHub (stable and nightly), compares with the current pinned version, lets the user pick a target version, and updates all version references across the repo. Use when the user wants to bump or upgrade flashinfer.

NVIDIA/TensorRT-LLM

Skill Claude CodeCodex ✓ vendor

Onboard a HuggingFace multimodal model (vision/audio/video + text) to the TensorRT-LLM PyTorch backend. Use when writing a new tensorrtllm/torch/models/modeling .py plus its input processor and weight mapper, or extending an existing VLM. Not for AutoDeploy — use ad-model-onboard for that path.

NVIDIA/TensorRT-LLM

Skill Claude CodeCodex ✓ vendor

Review, design, and refactor TensorRT-LLM PyTorch MoE code for architecture fit, clean code, maintainability, and testability. Always use for any modification, review, refactor, or design planning that touches MoE modules, including tensorrtllm/torch/moe/fusedmoe, ConfigurableMoE, MoE backends…

NVIDIA/TensorRT-LLM

Skill Claude CodeCodex ✓ vendor

Generate a source-backed starting trtllm-serve --config YAML for basic aggregate single-node PyTorch serving, aligned with checked-in TensorRT-LLM configs and deployment docs. Preserves explicit latency / balanced / throughput objectives. Excludes disaggregated, multi-node, and non-MTP speculative configs.

NVIDIA/TensorRT-LLM

Skill Claude CodeCodex ✓ vendor

Build Slurm scripts or Docker commands for TensorRT-LLM workloads. Resolves all parameters (docker image, mounts, parallelism, MPI mode), generates the complete script from Category templates, and writes both the script and a jobspec.json manifest to the work directory.

NVIDIA/TensorRT-LLM

Skill Claude CodeCodex ✓ vendor

Runs model-level and module-level tests for TensorRT-LLM. First classifies the test scope (module test or model test), then dispatches to the appropriate workflow. Model tests are further classified by type (functionality/smoke test, benchmark, or evaluation). Prompts the user for parallelism parameters (tp, ep, dp)…