An AI assistant for Comet Opik, a system for tracking and evaluating applications that use large language models. It covers prompts, projects, traces, metrics, experiments, and integrations with the Opik MCP server.

Use it to set up Opik, configure workspaces and projects, integrate the client, manage prompt versions, and inspect traces, metrics, and experiments.

It helps organize LLM application monitoring and investigate recorded runs while keeping prompt versions and access details managed.

Comet Opik — Agent for Claude Code by KIMISKI33

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

Clone the repo

git clone --depth 1 https://github.com/KIMISKI33/awesome-copilot

Made for: Claude Code.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

README.md

[![agentmods](https://agentmods.dev/badge/agents/kimiski33/awesome-copilot/comet-opik.svg)](https://agentmods.dev/agents/kimiski33/awesome-copilot/comet-opik)

Your own site

<a href="https://agentmods.dev/agents/kimiski33/awesome-copilot/comet-opik"><img src="https://agentmods.dev/badge/agents/kimiski33/awesome-copilot/comet-opik.svg" alt="Measured on agentmods" height="20"></a>

Per session 38 Only the description is in the session, so the agent can decide to use it. The body loads when it is invoked.

When invoked 2,469 The whole file, excluding the scripts and references it only reads on demand.

Security scan A 2 findings. A grade says what 26 rules found in the file — not that it is safe.

Origin 100% copy Near-identical to another mod in the catalogue.

Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

Model	Per session	Once invoked
Fable 5.1	$0.00038	$0.02469
Opus 5	$0.00019	$0.01234
Sonnet 5	$0.00008	$0.00494
Haiku 4.5	$0.00004	$0.00247

Measured 7d ago against content hash 9e94d6ed183c, method: parsed. Prices are Anthropic first-party input rates as of 2026-09-07, from the pricing page.

Security

Grade A, and why

Comet Opik scanned grade A with 2 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 7d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Unrestricted tool accesslowExcessive agency

A wildcard tool grant or "run any command" leaves no least-privilege boundary at all.

tools: ['*']

Downgraded: this mod is about security review, or the phrase is quoted, so it is likely naming the pattern rather than instructing it.

Makes network callslowCapability

Not a fault in itself. Listed so you know the mod talks to something, and to what.

- For scripted diagnostics, prefer CLI over raw HTTP. When CLI is unavailable (minimal containers/CI), replicate the requests with `curl`:

Origin

This is a copy

100% identical to Comet Opik — 0 lines differ, which has more behind it and is treated as the original. This page carries a canonical link to it rather than competing with it.

agents/comet-opik.agent.md · 173 lines

How it starts

The opening of the file, as written. The whole thing — 173 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Comet Opik Operations Guide

You are the all-in-one Comet Opik specialist for this repository. Integrate the Opik client, enforce prompt/version governance, manage workspaces and projects, and investigate traces, metrics, and experiments without disrupting existing business logic.

Prerequisites & Account Setup

User account + workspace
- Confirm they have a Comet account with Opik enabled. If not, direct them to https://www.comet.com/site/products/opik/ to sign up.
- Capture the workspace slug (the <workspace> in https://www.comet.com/opik/<workspace>/projects). For OSS installs default to default.
- If they are self-hosting, record the base API URL (default http://localhost:5173/api/) and auth story.
API key creation / retrieval
- Point them to the canonical API key page: https://www.comet.com/opik/<workspace>/get-started (always exposes the most recent key plus docs).
- Remind them to store the key securely (GitHub secrets, 1Password, etc.) and avoid pasting secrets into chat unless absolutely necessary.
- For OSS installs with auth disabled, document that no key is required but confirm they understand the security trade-offs.
Preferred configuration flow (opik configure)
- Ask the user to run:
```
pip install --upgrade opik
opik configure --api-key <key> --workspace <workspace> --url <base_url_if_not_default>
```
- This creates/updates ~/.opik.config. The MCP server (and SDK) automatically read this file via the Opik config loader, so no extra env vars are needed.
- If multiple workspaces are required, they can maintain separate config files and toggle via OPIK_CONFIG_PATH.
Fallback & validation
- If they cannot run opik configure, fall back to setting the COPILOT_MCP_OPIK_* variables listed below or create the INI file manually:
```
[opik]
api_key = <key>
workspace = <workspace>
url_override = https://www.comet.com/opik/api/
```
- Validate setup without leaking secrets:
```
opik config show --mask-api-key
```
  or, if the CLI is unavailable:
```
python - <<'PY'
from opik.config import OpikConfig
print(OpikConfig().as_dict(mask_api_key=True))
PY
```
- Confirm runtime dependencies before running tools: node -v ≥ 20.11, npx available, and either ~/.opik.config exists or the env vars are exported.

Read the full file on GitHub · 173 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

7d ago First seen · 173 lines · 38 tokens per session scan A 9e94d6ed183c

Subscribe to this mod's changes

Comet Opik is an agent published in the GitHub repository KIMISKI33/awesome-copilot (1 stars, last pushed yesterday), licensed MIT. It adds 38 tokens to every session and 2,469 once invoked, about $0.0002 per session on Opus 5. A static security scan graded it A with 2 findings (unrestricted tool access, makes network calls). It is 100% identical to Comet Opik, differing in 0 lines, and is treated as a copy.

Analytics and metrics Data and AI

Prompt Builder

Expert prompt engineering and validation system for creating high-quality prompts - Brought to you by microsoft/edge-ai.

github/awesome-copilot · 24 tokens

Research Harness Engineer

Research harness engineer for experiment campaigns: builds evaluation harnesses that are hard to fool, then keeps every reported number honest - null models first, calibration/held-out separation, baseline reproduction before improvement claims, paired error bars, and guards verified by deliberate breakage.

github/awesome-copilot · 56 tokens

fit

Selects algorithms, tunes hyperparameters, and builds reproducible training pipelines from baseline to production. Use when choosing a model architecture, designing a tuning strategy, or auditing training code for leakage and reproducibility. Trigger with "design training pipeline", "tune model hyperparameters".

jeremylongshore/tons-of-skills-marketplace · 57 tokens

mlops-engineer

ML operations agent for experiment tracking, model registry, feature stores, ML pipelines, model serving, drift monitoring, and AIOps.

pjt222/agent-almanac · 31 tokens

migration-reviewer

Use this agent after aidp-migrate-job completes to review a migrated .ipynb for correctness (NOT just "did it run"). Catches latent issues the cell-execute loop missed — wrong write-mode, lost rows, dropped columns, hardcoded paths, dead Databricks-isms. Outputs a structured review report.

ahmedawan-oracle/claude-code-plugins · 70 tokens

nn-embedding-expert

Embedding trained neural networks and tree ensembles as MINLP constraints via discopt.nn - OMLT-style full-space and reduced-space formulations, ReLU big-M, interval bound propagation, ONNX reader. Use when a trained ML surrogate must live inside an optimization problem.

jkitchin/discopt · 59 tokens

Comet Opik