Harbor is a framework for evaluating and improving AI agents and language models across reusable benchmarks and execution environments. Developers and researchers use it to run agent evaluations, create benchmarks, perform parallel experiments, and generate rollouts for reinforcement-learning optimization, with the catalogue offering skills, instructions, and an MCP integration for the framework.
Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add harbor-framework/harbor --skill upload-parity-experimentsgit clone --depth 1 https://github.com/harbor-framework/harborWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/harbor-framework/harbor/upload-parity-experiments)<a href="https://agentmods.dev/skills/harbor-framework/harbor/upload-parity-experiments"><img src="https://agentmods.dev/badge/skills/harbor-framework/harbor/upload-parity-experiments/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/harbor-framework/harbor/upload-parity-experiments"><img src="https://agentmods.dev/badge/skills/harbor-framework/harbor/upload-parity-experiments.svg" alt="Reviewed on agentmods" width="80" height="20"></a>- NVIDIA SkillSpector pass
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00049 | $0.01644 |
| Opus 5.5 | $0.00020 | $0.00658 |
| Sonnet 5.5 | $0.00010 | $0.00329 |
| Haiku 4.5 | $0.00005 | $0.00164 |
Grade A, and why
upload-parity-experiments scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 1mo ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
Copies of this mod
2 near-identical copies found in the catalogue:
- upload-parity-experiments — 100% identical, 0 lines differ
- upload-parity-experiments — 100% identical, 0 lines differ
How it starts
The opening of the file, as written. The whole thing — 176 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Upload Parity Experiments
Use this skill to publish Harbor parity experiment outputs to the shared Hugging Face dataset and capture the resulting discussion URL for the adapter's parity_pr field.
Why This Skill Exists
hf upload-large-foldercan be slow or unreliable for large parity bundles because it pushes through the Hub API commit loop.- A normal git clone of
harborframework/parity-experimentsis too expensive because the dataset is very large. - Hugging Face dataset PR refs are different from GitHub PR refs and are easy to misuse.
- Files larger than 10 MiB must be Git LFS-tracked before pushing.
This skill avoids the full clone by fetching only the target PR ref with --depth 1 --filter=blob:none and checking out only the paths needed for the current adapter.
Prereqs
- Ensure Hugging Face authentication is available with discussion-write permission. Either a classic
writetoken or a fine-grained token with globaldiscussion.writeenabled at https://huggingface.co/settings/tokens. A read-only or narrowly-scoped token will causecreate_pr.pyto fail with HTTP 403. - Keep the target dataset fixed to
harborframework/parity-experimentsunless the user explicitly asks for another repo. - Accept any local upload source that already contains the final files the user wants to publish.
Preferred Workflow
- Create or reuse a dataset PR.
- Prepare a sparse local worktree for that PR ref.
- Copy the local parity results into the sparse checkout.
- Ensure every file larger than 10 MiB is Git LFS-tracked before committing.
- Push directly to the PR ref with raw
git push. - Share the discussion URL and record it as the adapter's
parity_pr.
For large parity bundles, prefer raw git over hf upload-large-folder. The raw git path is materially faster and more reliable because it avoids the API-side commit loop and does not require cloning the entire parity dataset.
1. Create Or Reuse A Dataset PR
If the user already has a parity PR number, reuse it.
What ships with it
1 file beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 1mo ago First seen · 176 lines · 49 tokens per session scan A 257567d704bf
upload-parity-experiments is a skill published in the GitHub repository harbor-framework/harbor (6,185 stars, last pushed today), licensed Apache-2.0. It adds 49 tokens to every session and 1,644 once invoked, about $0.0002 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
tracely
Instrument AI agents with Tracely and turn their production traces into CI gates. Use when the user mentions Tracely, tracely-ai, tracelysdk, the tracely CLI, or asks to trace/observe an AI agent, add LLM evaluators or LLM-as-a-judge columns, debug why a trace or conversation isn't showing up, wire agent regression…
building-agents
Use when building or restructuring an LLM agent — provider adapter, tool calling, structured output, RAG, agent loop, eval gate, cost routing, tracing, MCP server — model-agnostic across OpenAI/Anthropic/Gemini/OSS so a model swap is a config change. NOT vector-store SQL alone (that is postgresdb) or service…
agent-eval
Use when measuring whether an LLM or agent system actually got better and gating merges on it: golden sets, fixing an inflated LLM-as-judge, scoring RAG (faithfulness, contextual recall) or agent trajectories (tool correctness, completion), or picking an eval framework. NOT building the agent loop, tools or RAG…
prompt-engineering
Use when one prompt must give the same right answer across reruns, models, and pasted-in hostile input: forcing a fixed schema, picking the few-shot set, ordering the prompt blocks, or the inline cases you run while tuning. NOT the agent loop, tools, or retrieval (that is building-agents), NOT a standing CI eval…
AI System Quality Engineer
Test LLM, RAG, MCP, and agentic systems end to end. Build golden datasets, run deterministic checks and LLM judges, score retrieval, probe prompt injection, verify tool use, and gate CI on thresholds. Orchestrates DeepEval, Ragas, promptfoo, and Langfuse.
eval-engine
Use this skill when a user supplies an AI feature spec, PRD, PMOS/task contract, existing eval suite, traces, outputs, state evidence, or release question and wants to define good, create or run evals, grade outcomes, agent trajectories, end-to-end system checkpoints, or promised memory behavior, add deterministic or…