parity-testing

parity-testing is a skill for Claude Code, Codex from NVIDIA-NeMo/Megatron-Bridge. It costs 69 tokens per session (2,145 once invoked), scanned A, original, Apache-2.0.

A testing guide for checking that model weights and outputs match between Hugging Face and Megatron Core, two machine-learning model systems.

In plain words
What is it for?
It helps run single- or multi-GPU weight round-trip tests, compare model logits, verify weights in Python, and perform text or vision-language generation checks.
Why use it?
It helps identify whether a model conversion changed weights or predictions, and separates exact weight checks from output and text-generation checks.

Skill for Claude CodeCodex

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add skills/nvidia-nemo/megatron-bridge/parity-testing
Any agent
npx skills add NVIDIA-NeMo/Megatron-Bridge --skill parity-testing
Clone the repo
git clone --depth 1 https://github.com/NVIDIA-NeMo/Megatron-Bridge

Made for: Claude Code, Codex.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for parity-testing

README.md
[![agentmods](https://agentmods.dev/badge/skills/nvidia-nemo/megatron-bridge/parity-testing.svg)](https://agentmods.dev/skills/nvidia-nemo/megatron-bridge/parity-testing)
Your own site
<a href="https://agentmods.dev/skills/nvidia-nemo/megatron-bridge/parity-testing"><img src="https://agentmods.dev/badge/skills/nvidia-nemo/megatron-bridge/parity-testing.svg" alt="Measured on agentmods" height="20"></a>
Per session 69 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 2,145 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00069 $0.02145
Opus 5 $0.00034 $0.01073
Sonnet 5 $0.00014 $0.00429
Haiku 4.5 $0.00007 $0.00215

Measured yesterday against content hash 858d41e321d4, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

parity-testing scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured yesterday.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

skills/parity-testing/SKILL.md · 201 lines

How it starts

The opening of the file, as written. The whole thing — 201 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Parity Testing for Megatron Bridge

This skill provides the decision framework for choosing the right verification tool and interpreting results. For the full model onboarding workflow (which includes parity testing as milestones 1 and 2), see the add-model-support skill.

Quick Decision: Which Tool to Run

What you want to verify Tool GPU? When to use
All weights round-trip exactly (single GPU) hf_megatron_roundtrip.py No First check after writing a bridge
Weights round-trip with TP/PP/EP hf_megatron_roundtrip_multi_gpu.py Yes After single-GPU passes
Forward-pass logit correlation compare_hf_and_megatron/compare.py Yes After round-trip passes
Text generation sanity hf_to_megatron_generate_text.py Yes Separate inference evidence
Programmatic weight check weights_verification_table() Yes Inside Python scripts
VLM generation sanity hf_to_megatron_generate_vlm.py Yes VLM models

All tools live under examples/conversion/.

3-Level Test Strategy

Level 1: State Dict Round-Trip (exact match)

The fastest and most fundamental check. If mappings can't perfectly round-trip weights, nothing else will work.

# Single-GPU round-trip
uv run python examples/conversion/hf_megatron_roundtrip.py \
    --hf-model-id <org>/<model>

# Multi-GPU with TP=2
uv run python -m torch.distributed.run --nproc_per_node=2 \
    examples/conversion/hf_megatron_roundtrip_multi_gpu.py \
    --hf-model-id <org>/<model> --tp 2

# Multi-GPU with PP=2
uv run python -m torch.distributed.run --nproc_per_node=2 \
    examples/conversion/hf_megatron_roundtrip_multi_gpu.py \
    --hf-model-id <org>/<model> --pp 2

Expected: Every weight shows "Matches Original: checkmark". Any "X" means the param mapping has an error.

Tolerance: Exact match (max_diff == 0.0). Round-trip conversions are pure tensor reshaping — no floating-point arithmetic is involved.

For programmatic verification inside scripts, use the built-in verifier:

Read the full file on GitHub · 201 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. yesterday Changed · +9 lines 858d41e321d4
  2. 5d ago First seen · 192 lines · 69 tokens per session scan A 7ed8b6298d81

Subscribe to this mod's changes

parity-testing is a skill published in the GitHub repository NVIDIA-NeMo/Megatron-Bridge (900 stars, last pushed today), licensed Apache-2.0. It adds 69 tokens to every session and 2,145 once invoked, about $0.0003 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Related

Other skills, from other repositories

agent-platform-rag-engine-management

Manage and query Agent Platform RAG Engine Corpora and retrieve grounded contexts using the Google GenAI SDK. Use when listing RAG corpora or files, inspecting a corpus, retrieving contexts, or generating content grounded in a RAG corpus. Do not use for standard database queries (use SQL/Spanner skills), Google…

google/skills · 85 tokens

agent-platform-model-registry

Agent Platform Model Registry Management. Use when you need to upload, list, describe, update, or delete machine learning models (and their versions) in the Agent Platform Model Registry. Don't use for model training, model deployment to endpoints, or managing non-Agent Platform models.

google/skills · 60 tokens

foundry-config-setup

Resolve missing setup caused by a hardcoded Foundry project endpoint or model in a sample. Use when a sample fails because it uses a placeholder/hardcoded projectendpoint (for example "https://your-project.services.ai.azure.com") or a hardcoded model instead of reading them from the environment.

microsoft/agent-framework · 65 tokens

google-cloud-solution-agentic-analytics-spark-knowledge-catalog

Discovers requirements and generates guidance to design and deploy a governed, secure agentic-analytics solution for data that's distributed across Google Cloud, other cloud providers, or on-premises. Data that's outside Google Cloud (such as data from Databricks, Snowflake, Salesforce, SAP, or Oracle systems) is…

google/skills · 138 tokens

training-check

Interactively monitor training metrics from the current Codex session, periodically checking WandB or fallback logs for NaN, divergence, plateaus, and broken runs.

wanshuiyin/Auto-claude-code-research-in-sleep · 35 tokens

nemo-automodel-launcher-config

Configure NeMo AutoModel job launches for interactive runs, Slurm clusters, and SkyPilot cloud execution.

NVIDIA/skills · 30 tokens