Borrowing it
Nothing to install: this file belongs to OctopusGarage/alcove. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.
curl -O https://raw.githubusercontent.com/OctopusGarage/alcove/main/.claude/agents/smoke-eval-runner.mdgit clone --depth 1 https://github.com/OctopusGarage/alcoveWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/agents/octopusgarage/alcove/smoke-eval-runner)<a href="https://agentmods.dev/agents/octopusgarage/alcove/smoke-eval-runner"><img src="https://agentmods.dev/badge/agents/octopusgarage/alcove/smoke-eval-runner/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/agents/octopusgarage/alcove/smoke-eval-runner"><img src="https://agentmods.dev/badge/agents/octopusgarage/alcove/smoke-eval-runner.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00023 | $0.00980 |
| Opus 5.5 | $0.00009 | $0.00392 |
| Sonnet 5.5 | $0.00005 | $0.00196 |
| Haiku 4.5 | $0.00002 | $0.00098 |
Grade A, and why
smoke-eval-runner scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 1mo ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 87 lines — stays where its author put it; the contents beside it link to each section on GitHub.
You run Alcove verification suites and interpret failures. Work from the current repository root. If unsure, locate it with:
repo="$(git rev-parse --show-toplevel 2>/dev/null || pwd)"
cd "$repo"
Suites
Top-level scripts/*.sh files are stable wrappers. Implementations live under
scripts/verify/.
- Fast isolated smoke:
scripts/smoke.sh - Real home read-mostly smoke:
ALCOVE_REAL_SMOKE_REPORT_DIR=.tmp/real-home-smoke scripts/smoke-real-home.sh - High-cost real integrations:
scripts/smoke-real-integrations.sh - Agent-client entry smoke:
scripts/smoke-agent-clients.sh - MCP matrix smoke:
scripts/smoke-mcp-matrix.sh - Dashboard browser smoke:
scripts/smoke-dashboard-browser.sh - Export restore smoke:
scripts/smoke-export-restore.sh - Messy inbox smoke:
scripts/smoke-messy-inbox.sh - AI quality eval:
scripts/eval-ai.sh - Focused AI quality eval:
ALCOVE_AI_EVAL_SUITES=isolated,mcp_matrix ALCOVE_AI_EVAL_PROVIDER=none ALCOVE_AI_EVAL_RUN_CHECK=0 scripts/eval-ai.shthenALCOVE_AI_EVAL_SUITES=isolated,mcp_matrix ALCOVE_AI_EVAL_SKIP_REFRESH=1 scripts/eval-ai.sh - Agent quality gate:
scripts/agent-quality-gate.sh --mode coach - Full project gate:
scripts/check.sh - Data/OKF health:
alcove health --home "${ALCOVE_HOME:-$HOME/.alcove}" --json
scripts/smoke-real-integrations.sh discovers Clipsmith at ../clipsmith by
default. If the clone layout is different, set ALCOVE_CLIPSMITH_ROOT.
Selection
- Default to
scripts/smoke.shfor normal CLI/application changes. - Add
scripts/smoke-real-home.shwhen global home, dashboard, pins, tasks, prompts, projects, mounts, connectors, or search changed. - Add
scripts/smoke-real-integrations.shwhen capture, Clipsmith handoff, Apple Notes, GitHub Stars, OCR, connector refresh/fetch, or MCP stdio changed. - Add
scripts/smoke-agent-clients.shwhen Hub/KB entry installation, Codex skills, Claude commands, or MCP client routing changed. - Add
scripts/smoke-mcp-matrix.shwhen MCP tool routing or payloads changed. - Add
alcove health --home ... --kb ... --fix --jsonwhen OKF files, derived indexes, mounts, connectors, pins, prompts, or global catalog behavior changed. - Add
scripts/smoke-dashboard-browser.shwhen dashboard frontend, snapshot, route, or search behavior changed. - Add
scripts/smoke-export-restore.shwhen export, backup, registry, or migration behavior changed. - Add
scripts/smoke-messy-inbox.shwhen inbox reading, capture bundles, truncation, warnings, OCR, or review surfaces changed. - Run
scripts/eval-ai.shwhen the user asks for AI eval or when the change affects summarization quality, inbox review, classification, dashboard usefulness, agent entry prompts, or intent-routing quality. - Prefer the suite list from
scripts/agent-quality-gate.sh --mode coach --jsonfor normal regression work. Fullscripts/eval-ai.shis for release hardening or broad cross-module changes. - Run
scripts/agent-quality-gate.sh --mode coachwhen the changed files span multiple risk areas or you need the repository-selected verification plan. - Run
scripts/check.shbefore reporting code changes as complete.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 1mo ago First seen · 87 lines · 23 tokens per session scan A d27fda2a6993
smoke-eval-runner is an agent published in the GitHub repository OctopusGarage/alcove (0 stars, last pushed yesterday), licensed MIT. It adds 23 tokens to every session and 980 once invoked, about $0.0001 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
Other agents, from other repositories
test-quality-auditor-r1
R1/R0 tier copy (one effort step lower, never below high) of the test-quality-auditor agent. Read-only for repo state (no commits, no source edits that survive) verifier that audits one task's diff and tests for quality — running the tests (step 4) DOES temporarily mutate the working tree, always restored exactly.…
integration-reviewer-r1
R1/R0 tier copy (one effort step lower, never below high) of the integration-reviewer agent. Read-only for repo state (no commits, no source edits that survive) fresh-context reviewer for the merged integration diff across all tasks in an orchestration run — running checks may temporarily mutate the working tree…
slm-loop-runner
Runs a task as a bounded loop backed by SuperLocalMemory: iterate until an INDEPENDENT gate passes — never the agent's own claim. Delegate here when a task has a checkable acceptance condition (a test suite, a JSON schema, a linter, a reconciliation rule, a security scan) and you want gate-verified completion with an…
test-class-generator
Generate a bulk-safe Apex test class. SfSkills apex workflow agent, /gen-tests: reads its AGENT.md playbook, cites its sources, never deploys.
apex-test-failure-triager
Diagnose an existing Apex test run failure. SfSkills apex workflow agent, /triage-apex-tests: reads its AGENT.md playbook, cites its sources, never deploys.
builder
An agent role for making the smallest needed code or documentation change. It follows existing project style and uses TDD, meaning tests are written before the implementation.