sonic

SONIC

When To Use

Use this skill for NVIDIA GEAR-SONIC whole-body-control workbench changes, including standalone training, export, evaluation, serving, image routing, and workflow composition with retargeting or MJLab.

Procedure

1. Confirm the current CLI surface before editing:

   npa workbench sonic --help
   

2. Use train for policy training, export for export artifacts, eval for evaluation, and serve for runtime serving. Use deploy, status, and list for operational lifecycle.

3. Route first-party SONIC images through npa/src/npa/deploy/sonic_image_manifest.json; do not hardcode image tags in workflows or docs.

4. Keep S3 paths run-scoped so retraining and re-evaluation do not overwrite previous artifacts.

Three-Tier Contract

• CLI: deploy, train, export, eval, serve, status, and list.
• SDK/API: keep train/eval/export/serve request construction shared with service payloads and tests where possible.
• Workflow: SONIC specs live under workflows/testing/ (sonic-train.yaml, sonic-export.yaml, sonic-eval.yaml, sonic-export-eval.yaml, sonic-locomotion-finetuning.yaml). Submit them with npa workbench workflow submit. sonic export / sonic eval accept s3:// checkpoints, ONNX policies and outputs, so no staging glue is needed. Raw SkyPilot coverage for the submit wrapper lives in npa/tests/fixtures/skypilot/; shipped SONIC workflow authoring stays on the npa.workflow specs.

Routing And Validation

• The legacy baked L40S and inherited MuJoCo variants are quarantined because their built bytes contain restricted NVIDIA payloads; resolvers must reject them.
• sonic-mujoco-runtime-fetch is an independently rebuilt public release on a digest-pinned Python base and hash-locked CUDA Toolkit/MuJoCo closure. It is not derived from either quarantined image; release promotion is bound to its exact clean, GPU-accepted public development digest.
• Use the active host-mounted runtime-fetch image selected by npa/src/npa/deploy/sonic_image_manifest.json for RTX PRO 6000 Blackwell Kubernetes targets with NVIDIA GPU Operator mounted drivers. A B300 validation image uses the same Dockerfile but must be supplied explicitly by immutable digest until its own accepted release updates the manifest. B300 is compute capability 10.3: that development image must use the digest-pinned CUDA 13 base, PyTorch cu130, CUDA 13 NVRTC, and the truthful sm80-sm90-sm100-sm103-sm120 target contract. The official cu130 PyTorch wheel exposes Blackwell 10.x family SASS as sm_100; the literal sm_103 requirement applies to CUDA 13 NVRTC JIT compilation on the B300 device. CUDA 12.8 NVRTC rejects sm_103; do not treat environment startup before that JIT boundary as B300 training evidence.
• SONIC render validation requires RT-capable GPUs. Use RTX PRO 6000 Blackwell; do not silently fall back to the quarantined H100/L40S images.
• sonic eval --backend container consumes ONNX plus its metadata sidecar and resolves isaac-render, including explicit manifest variants. The separate mujoco-eval container entrypoint consumes a checkpoint through SONIC_EVAL_CHECKPOINT_PATH; it is not a substitute for the ONNX contract. Image resolution must use the workload at every caller that knows its stage.
• Generic Blackwell labels containing B200/B300 remain datacenter targets. Custom workflow --image overrides may select an unpublished runtime without being attributed to a first-party variant; callers must validate those bytes.
• The built-in serverless compute-only default is intentionally unavailable: its L40S/H100/H200 images are quarantined, while the active image requires Kubernetes GPU Operator driver mounts. A serverless run must supply an independently validated compute-only --image, or fail before provisioning.