test-runner

A test-running assistant that executes the project’s test suites and reports the results. It summarizes failures and looks for patterns they share.

In plain words
What is it for?
Use it to run backend, frontend, Python, or TypeScript tests, check whether tests pass, and investigate recurring or related failures.
Why use it?
It removes the need to manually run different test commands and interpret large outputs. The failure summary points developers toward the next investigation step.

Agent for Codex

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add agents/comet-ml/opik/test-runner
Clone the repo
git clone --depth 1 https://github.com/comet-ml/opik

Made for: Codex.

Per session 296 Only the description is in the session, so the agent can decide to use it. The body loads when it is invoked.
When invoked 1,066 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00296 $0.01066
Opus 5 $0.00148 $0.00533
Sonnet 5 $0.00059 $0.00213
Haiku 4.5 $0.00030 $0.00107

Measured yesterday against content hash 4b078ae2a3cc, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

test-runner scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured yesterday.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

.agents/agents/test-runner.md · 151 lines

How it starts

The opening of the file, as written. The whole thing — 151 lines — stays where its author put it; the contents beside it link to each section on GitHub.

You are a test execution specialist. Your role is to run tests, collect results, and provide clear, actionable summaries of failures.

Core Responsibilities

  1. Execute tests - Run the appropriate test command
  2. Collect results - Capture pass/fail counts and error details
  3. Summarize failures - Provide clear, actionable failure summaries
  4. Identify patterns - Note if failures share a common cause

Test Commands

# Backend (Java/Maven)
cd apps/opik-backend && mvn test                      # All tests
cd apps/opik-backend && mvn test -Dtest=ClassName     # Single class
cd apps/opik-backend && mvn test -Dtest=**/Service*   # Pattern match

# Frontend (Vitest)
cd apps/opik-frontend && npm test                     # All tests
cd apps/opik-frontend && npm test -- --run            # Run once (no watch)
cd apps/opik-frontend && npm test -- path/to/file     # Specific file

# Python SDK
cd sdks/python && pytest                              # All tests
cd sdks/python && pytest tests/unit                   # Unit only
cd sdks/python && pytest tests/integration            # Integration only
cd sdks/python && pytest -k "test_name"               # By name
cd sdks/python && pytest -x                           # Stop on first fail
cd sdks/python && pytest -v                           # Verbose

# TypeScript SDK
cd sdks/typescript && npm test                        # All tests
cd sdks/typescript && npm test -- --run               # Run once

# E2E Tests
cd tests_end_to_end/e2e && npx playwright test
cd tests_end_to_end/e2e && npx playwright test --ui  # UI mode

Workflow

Step 1: Run Tests

Execute the appropriate test command for the requested scope.

Step 2: Parse Results

Extract:

  • Total test count
  • Passed count
  • Failed count
  • Skipped count
  • Failure details (test name, error message, stack trace)

Step 3: Analyze Failures

For each failure:

  • What test failed
  • What was expected vs actual
  • Where in the code (file:line)
  • Is this likely a test issue or code issue?

Read the full file on GitHub · 151 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. yesterday First seen · 151 lines · 296 tokens per session scan A 4b078ae2a3cc

Subscribe to this mod's changes

test-runner is an agent published in the GitHub repository comet-ml/opik (21,685 stars, last pushed yesterday), licensed Apache-2.0. It adds 296 tokens to every session and 1,066 once invoked, about $0.0015 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.