Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add comet-ml/opik-skills --skill opik-optimizegit clone --depth 1 https://github.com/comet-ml/opik-skillsWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/comet-ml/opik-skills/opik-optimize)<a href="https://agentmods.dev/skills/comet-ml/opik-skills/opik-optimize"><img src="https://agentmods.dev/badge/skills/comet-ml/opik-skills/opik-optimize/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/comet-ml/opik-skills/opik-optimize"><img src="https://agentmods.dev/badge/skills/comet-ml/opik-skills/opik-optimize.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00143 | $0.03098 |
| Opus 5.5 | $0.00057 | $0.01239 |
| Sonnet 5.5 | $0.00029 | $0.00620 |
| Haiku 4.5 | $0.00014 | $0.00310 |
Grade A, and why
opik-optimize scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 22d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
This is a copy
98% identical to opik-optimize — 54 lines differ, which has more behind it and is treated as the original. This page carries a canonical link to it rather than competing with it.
How it starts
The opening of the file, as written. The whole thing — 143 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Optimize — Improve a Prompt Against Data
Definition of done: an optimized prompt whose score on the metric beats the baseline on data it was not tuned on, the optimization run link in Opik, the cost of getting there, and the winner saved as a new prompt version (when the prompt lives in the library) — with the swap into code left as the next step. If the optimization can't run within a stated budget, stop at the first genuine blocker and return exactly one next step. A prompt that scores higher only on its own training items is not an improvement.
Operate: measure the baseline first, state the budget before spending it, hold data out, pick the algorithm for the failure you see, save the winner where it can be versioned — and change no application code. The only file this skill writes is a runner outside the repo; the prompt is saved to Opik, not into the codebase.
Inputs
The entry point is /opik-optimize <prompt-name> (a prompt-library prompt), /opik-optimize <path or function> (a prompt in code), or /opik-optimize (find the system prompt in this repo). Infer the rest; treat these as optional overrides:
- dataset (default: existing dataset for the project → export the regression suite → build from traces) · metric (default: heuristic on
expected_outputif present, else one binary judge) · algorithm (default:MetaPromptOptimizer) · budget (default:n_samples=50,max_trials=10) · model · validation split (default: hold out 20%).
Ask only at a genuine, non-inferable blocker (see Blockers).
Activation — the only in-scope work
1. Resolve the prompt
- Library:
client.get_chat_prompt(name)(orget_promptfor a text prompt). Note the currentversion— that is the baseline. - Code: grep for the system prompt /
messages=[...]; read it verbatim. Note where it lives; you will not edit it. - Trace: the
llmspan'sinput.messageson a representative trace.
The optimizer's opik_optimizer.ChatPrompt is a different class from the library's opik.ChatPrompt — build it from the raw messages yourself:
from opik_optimizer import ChatPrompt
prompt = ChatPrompt(name="<name>", system="<system text>", user="{question}") # or messages=[...]; {var} names must match dataset keys
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 22d ago First seen · 143 lines · 143 tokens per session scan A 008305d6028f
opik-optimize is a skill published in the GitHub repository comet-ml/opik-skills (8 stars, last pushed yesterday), licensed Apache-2.0. It adds 143 tokens to every session and 3,098 once invoked, about $0.0006 per session on Opus 5.5. A static security scan graded it A with 0 findings. It is 98% identical to opik-optimize, differing in 54 lines, and is treated as a copy.
Other skills, from other repositories
opik-optimize
Improve a prompt with the Opik Agent Optimizer — resolve the prompt, a dataset, and a metric, pick the algorithm, run a bounded optimization, check the gain on held-out data, and save the winner as a new prompt version with the optimization run link. Runs via the opik-optimizer package; reads prompts and datasets via…
opik-online-eval
Take a judge live on production traffic — create an Opik online evaluation rule (LLM-as-judge or Python metric) on a project with sampling, filters, variable mapping, and a cost cap, then confirm new traces are being scored. Works over the SDK's REST client; reads rules and score names via the MCP when connected.…
opik-test
Turn a failing Opik trace (or a described failure) into a repeatable regression check — a test-suite item with the trace's input and one or two binary assertions — so a fix can be verified by the compare skill. Works over the SDK; uses the MCP write tool when connected. Returns the suite, the item, and the assertion.…
opik-instrument
Add Opik tracing to an existing app and verify a real trace lands. Installs the Opik package, detects the language and LLM framework, adds the minimum tracing, runs a safe representative path, confirms a trace in Opik, and returns the trace link. Use for "instrument my code", "add opik tracing", "add observability"…
opik-evaluate
Build an LLM evaluation and run it against the app, returning an Opik experiment with scores and its link. Picks a test suite with judge assertions or a dataset with metrics, sources cases from traces or synthetic data, scores heuristics-first then one-failure-mode judges, runs client-side via the SDK or server-side…
llm-app-patterns
Production-ready patterns for building LLM applications. Covers RAG pipelines, agent architectures, prompt IDEs, and LLMOps monitoring. Use when designing AI applications, implementing RAG, building agents, or setting up LLM observability.