opik-mcp: Command for Claude Code

.claude/commands/dogfood.md

dogfood is a command for Claude Code from comet-ml/opik-mcp. It costs 23 tokens per session (953 once invoked), scanned A, original, Apache-2.0.

A command that tests the current branch's MCP server beside the main branch's server in a private, headless session. It reports differences a customer would notice and checks the relevant project flows without changing Claude configuration.

In plain words
What is it for?
Use it to compare a branch with the main branch, exercise customer-facing flows, and find release-blocking problems in an MCP server.
Why use it?
It provides a customer-style comparison before shipping while keeping the test servers isolated from other sessions. It also records the branch changes and existing project notes used during the test.

Command for Claude Code

Written for Claude Code: disable-model-invocation in frontmatter.

This is comet-ml/opik-mcp's own configuration. It tells Claude Code how to work on opik-mcp itself, so it is not a mod to install elsewhere. Copy it as a starting point and replace the rules that are about this project. Everything opik-mcp configures →

Needs its repository: it runs a file that does not travel with it, so clone the repository first. The line is python3 scripts/dev/install_branch.py dogfood-prepare [--workspace <ws>].

Reuse

Borrowing it

Nothing to install: this file belongs to comet-ml/opik-mcp. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.

Copy the file
curl -O https://raw.githubusercontent.com/comet-ml/opik-mcp/main/.claude/commands/dogfood.md
Clone the repo
git clone --depth 1 https://github.com/comet-ml/opik-mcp

Made for: Claude Code.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for dogfood

README.md
[![agentmods](https://agentmods.dev/badge/commands/comet-ml/opik-mcp/dogfood/github.svg)](https://agentmods.dev/commands/comet-ml/opik-mcp/dogfood)
Your own site
<a href="https://agentmods.dev/commands/comet-ml/opik-mcp/dogfood"><img src="https://agentmods.dev/badge/commands/comet-ml/opik-mcp/dogfood/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

agentmods 80×15 button for dogfood

Your own site · 80×15
<a href="https://agentmods.dev/commands/comet-ml/opik-mcp/dogfood"><img src="https://agentmods.dev/badge/commands/comet-ml/opik-mcp/dogfood.svg" alt="Reviewed on agentmods" width="80" height="20"></a>
Per session 23 Only the description is in the session, so the agent can decide to use it. The body loads when it is invoked.
When invoked 953 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00023 $0.00953
Opus 5.5 $0.00009 $0.00381
Sonnet 5.5 $0.00005 $0.00191
Haiku 4.5 $0.00002 $0.00095

Measured 24d ago against content hash fe440db79712, method: parsed. Prices are Anthropic first-party input rates as of 2026-10-07, from the pricing page.

Security

Grade A, and why

dogfood scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 24d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

.claude/commands/dogfood.md · 98 lines

How it starts

The opening of the file, as written. The whole thing — 98 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Test this branch the way a customer would use it, through a headless session that loads this branch's server next to main's, and tell me if it is ready to ship. Extra focus from me: $ARGUMENTS

If the branch doesn't touch src/, both servers run the same code and no differences is the expected result. Say that in one line and still report problems found on both.

1. Build the two servers

Nothing gets registered in any Claude config, so no restart is needed and no other session pays for these servers.

python3 scripts/dev/install_branch.py dogfood-prepare [--workspace <ws>]

This builds this branch and a fresh origin/main into their own venvs and writes a private MCP config naming them opik-branch and opik-base, on the same workspace. The config holds ${OPIK_API_KEY}, never the key.

2. Read what changed and what we already know

  • git diff origin/main...HEAD, the commit messages, and the ticket (OPIK- from the branch name, via Jira if available).
  • Every file in .claude/dogfood/memory/ (format in its README).

3. Pick the flows

Core flows, phrased the way a customer would ask:

  1. How is my project doing?
  2. What is broken in production?
  3. Compare these two experiments.
  4. Find the case where the judge disagreed.
  5. Show me the traces with errors from the last hour.
  6. Explain this trace.

Add 2 to 4 flows aimed straight at what the diff changed. The servers load only in the headless run, so describe what to pick ("the largest recent trace", "two experiments on one dataset") and let that session find real ids with list. Don't invent ids.

4. Run the flows in a headless session

Write the flows from step 3 to a prompt file under the scratchpad or /tmp, with these instructions for the headless session:

  • Make the same calls with the same arguments through opik-branch and opik-base. Answer each flow as a customer would expect, then compare: is the answer right for the data, do the UI links look right, what does it cost (the size header, branch vs base), does anything say more than the data supports or cut without saying so?
  • Put each difference in one bucket: intended (matches the diff or ticket), regression (worse on the branch), pre-existing (wrong on both), improvement (better, and not the point of the change).
  • Skip these accepted findings: <paste each accepted entry's symptom, unless a file under its recheck_when changed in this branch; then ask for a re-check and a "same" or "changed" answer>.
  • Read only. No write operations.
  • Return the table from step 5 and candidate memory entries.

Read the full file on GitHub · 98 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 24d ago First seen · 98 lines · 23 tokens per session scan A fe440db79712

Subscribe to this mod's changes

dogfood is a command published in the GitHub repository comet-ml/opik-mcp (223 stars, last pushed 2d ago), licensed Apache-2.0. It adds 23 tokens to every session and 953 once invoked, about $0.0001 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-25.