Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add comet-ml/opik-claude-code-plugin --skill opik-online-evalgit clone --depth 1 https://github.com/comet-ml/opik-claude-code-pluginWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/comet-ml/opik-claude-code-plugin/opik-online-eval)<a href="https://agentmods.dev/skills/comet-ml/opik-claude-code-plugin/opik-online-eval"><img src="https://agentmods.dev/badge/skills/comet-ml/opik-claude-code-plugin/opik-online-eval/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/comet-ml/opik-claude-code-plugin/opik-online-eval"><img src="https://agentmods.dev/badge/skills/comet-ml/opik-claude-code-plugin/opik-online-eval.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00146 | $0.02934 |
| Opus 5.5 | $0.00058 | $0.01174 |
| Sonnet 5.5 | $0.00029 | $0.00587 |
| Haiku 4.5 | $0.00015 | $0.00293 |
Grade A, and why
opik-online-eval scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 22d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
This is a copy
100% identical to opik-online-eval — 52 lines differ, which has more behind it and is treated as the original. This page carries a canonical link to it rather than competing with it.
How it starts
The opening of the file, as written. The whole thing — 137 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Online Eval — Take a Judge Live
Definition of done: an online evaluation rule exists on the project, is enabled, and has scored at least one new trace — confirmed by reading a fresh trace's feedback score, not by the create call returning. The rule scores one failure mode, samples at a rate the project's volume can afford, carries a cost cap, and maps its variables to the fields the traces actually have. If traffic hasn't arrived yet, the done state is "rule live, unverified — watch this filter". If the rule can't be created, stop at the first genuine blocker and return exactly one next step.
Operate: one failure mode per rule, sample before you scale, cap the spend, verify on a real trace — and change no application code. This skill writes to Opik only.
Inputs
The entry point is /opik-online-eval <what to score> ("hallucinations", "the refund-window answer", "refusals"), /opik-online-eval right after /opik-evaluate validated a judge (take that judge live), or /opik-online-eval <project>. Infer the rest; treat these as optional overrides:
- project (default: configured) · scope (default: trace;
spanfor one model call,threadfor whole conversations) · sampling rate (default: 1.0 under ~1k traces/day, else 0.1–0.2) · filters (default: none; typical: anenvironmentor tag filter) · judge model · cost cap (default: set one) · variable mapping (default:input → input,output → output).
Ask only at a genuine, non-inferable blocker (see Blockers).
Activation — the only in-scope work
1. Resolve project, judge, and scope
Confirm Opik is reachable (~/.opik.config or OPIK_API_KEY; otherwise → Blocker). Resolve the project id from a trace or by name. Decide the judge:
- A judge
/opik-evaluatealready validated → reuse its prompt and output schema verbatim. - A named metric (hallucination, answer relevance, moderation) → a minimal binary judge for that one failure mode (
../opik-evaluate/references/write-judge-prompt.md). - A mechanical check (JSON valid, contains a disclaimer, latency budget) → a Python metric rule, not a judge.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 22d ago First seen · 137 lines · 146 tokens per session scan A 0611da9b553d
opik-online-eval is a skill published in the GitHub repository comet-ml/opik-claude-code-plugin (23 stars, last pushed 7d ago), licensed Apache-2.0. It adds 146 tokens to every session and 2,934 once invoked, about $0.0006 per session on Opus 5.5. A static security scan graded it A with 0 findings. It is 100% identical to opik-online-eval, differing in 52 lines, and is treated as a copy.
Other skills, from other repositories
huggingface_hub
Discover open machine learning models on Hugging Face Hub via the public model search API.
claude-bridge
Use when orchestrating multiple Claude Code chats, reading other chats' history, sending messages between chats, or searching across chat sessions. Triggers on phrases like "peer chat", "sister chat", "other chat", "cross-chat", "what is the other agent doing", "find in chats", "manager agent", "worker agent"…
claude-bridge-role-manager
Playbook for a Claude Code agent in the role of orchestrator of 2-N worker peers over the claude-bridge MCP. Use when you have a team of peers and want to efficiently dispatch tasks, gate outputs, resolve conflicts, protect invariants, route memory, and hold the interface to the human owner. Triggers — "I'm the…
claude-bridge-role-memory-keeper
Playbook for a Claude Code agent in the dedicated memory keeper role — single-writer for shared agent memory across a team of 3+ peers. Use when you are designated as the keeper (= other peers send you write candidates, you write them and watch for drift). Memory hygiene + reconciliation against the canon + drift…
claude-bridge-setup
Guide the user through activating claude-bridge live-data hooks in /.claude/settings.json. Use when the user asks how to install the plugin's statusLine wrapper or PostToolUse refresh-limits hook, or when peercontextstatus / ratelimitstatus returns hasLiveData: false with a setupPointer. Triggers — "setup live data"…
prompt-optimizer
Analyze raw prompts, identify intent and gaps, match ECC components (skills/commands/agents/hooks), and output a ready-to-paste optimized prompt. Advisory role only — never executes the task itself. TRIGGER when: user says "optimize prompt", "improve my prompt", "how to write a prompt for", "help me prompt", "rewrite…