Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add NITISH-R-G/hackerrank-orchestrate-skills --skill orchestrate-confidence-calibratorgit clone --depth 1 https://github.com/NITISH-R-G/hackerrank-orchestrate-skillsWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/nitish-r-g/hackerrank-orchestrate-skills/orchestrate-confidence-calibrator)<a href="https://agentmods.dev/skills/nitish-r-g/hackerrank-orchestrate-skills/orchestrate-confidence-calibrator"><img src="https://agentmods.dev/badge/skills/nitish-r-g/hackerrank-orchestrate-skills/orchestrate-confidence-calibrator/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/nitish-r-g/hackerrank-orchestrate-skills/orchestrate-confidence-calibrator"><img src="https://agentmods.dev/badge/skills/nitish-r-g/hackerrank-orchestrate-skills/orchestrate-confidence-calibrator.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00043 | $0.00614 |
| Opus 5 | $0.00022 | $0.00307 |
| Sonnet 5 | $0.00009 | $0.00123 |
| Haiku 4.5 | $0.00004 | $0.00061 |
Grade A, and why
orchestrate-confidence-calibrator scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 11d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 56 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Orchestrate: Confidence Calibrator
Evidence tier: first-hand build (August 2026). Grounded in a completed Orchestrate submission that was audited to destruction — 48 logged defects, 9 measured-and-rejected optimisations, 17 certification scripts. Every number below was measured on that system. Nothing here claims access to HackerRank's internal scoring.
The rule
Match the target, not your intuition about the target.
Confidence is an explicitly scored dimension. It is graded against the labels, not against an abstract notion of good calibration.
What this caught in a real build
Dynamic confidence lost to a constant. "More matched signals should mean higher confidence" is obviously right. Measured: MAE 0.0287 dynamic vs 0.0263 static — worse overall and worse on every action subset. It also emitted values outside the observed band on 3/30 rows, masked only by a clamp. Deleted, not disabled.
The ECE trap. The system showed Expected Calibration Error of 0.138 — systematically under-confident. The textbook fix is obvious. Before applying it, one question: what is the ground truth's own ECE?
Answer: 0.1597 — worse. The labels are deliberately under-confident. Every ECE-improving shift made error against the actual target strictly worse (0.0263 → 0.1467).
Optimising a textbook metric would have moved the system away from the thing being scored.
How to set values
- Read the confidence column of the labeled samples. Note the observed band.
- Clamp your output to that band. A value outside it is provably unlike any label.
- Assign a value per decision class, then measure MAE against the labeled rows.
- Only adopt a dynamic scheme if it beats the constant. Measure; do not assume.
The checklist
- Observed confidence band extracted from the labeled data
- Output clamped to that band
- MAE measured against ground-truth confidence values
- Any dynamic scheme benchmarked against a static baseline before adoption
- Ground truth's own ECE computed before "fixing" your calibration
- Rejected schemes recorded with the number that killed them
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 11d ago First seen · 56 lines · 0 tokens per session scan A 7806463ff182
orchestrate-confidence-calibrator is a skill published in the GitHub repository NITISH-R-G/hackerrank-orchestrate-skills (3 stars, last pushed 1mo ago), licensed MIT. It adds 43 tokens to every session and 614 once invoked, about $0.0002 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
Other skills, from other repositories
general
Handle everyday conversation, answer questions, manage files, take notes, run scripts, and maintain persistent memory across sessions. Use when the user asks a general question, requests file operations, wants to brainstorm ideas, needs to-do tracking, asks you to remember something, or requests skill search and…
multi-bot
Coordinates responses between multiple GolemBot instances in a shared fleet. Use when the bot operates in a group chat with other bots, needs to decide whether to respond or pass, or must call a peer bot's API to fetch cross-domain data.
kb-guide
Search, read, create, and update knowledge base entries via MCP-connected KB tools. Use when the user asks to look up documentation, find existing articles, check if docs exist on a topic, create a new KB entry, update an existing document, or when domain questions should be answered from the knowledge base first.
ops
Content operations assistant — drafts blog posts, social media copy, and marketing materials, compiles data briefings, and tracks competitor activity. Use when the user asks to write a blog post, draft social media content, create marketing copy, generate a weekly report, compile operational metrics, update the…
escalation
Escalate unresolvable or sensitive requests to a human agent by recording an escalation entry. Use when the user asks to speak to a human, the bot cannot answer confidently, the request involves financial, legal, or security concerns, a safety issue is detected, or the user is frustrated after repeated failures.
code-review
Reviews code changes, pull requests, and diffs for correctness, security, performance, and style. Use when the user submits a PR for review, asks to review a diff or code snippet, or requests a quality check on recent changes.