Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add mitodl/agent-kit --skill run-canary-locallygit clone --depth 1 https://github.com/mitodl/agent-kitWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/mitodl/agent-kit/run-canary-locally)<a href="https://agentmods.dev/skills/mitodl/agent-kit/run-canary-locally"><img src="https://agentmods.dev/badge/skills/mitodl/agent-kit/run-canary-locally/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/mitodl/agent-kit/run-canary-locally"><img src="https://agentmods.dev/badge/skills/mitodl/agent-kit/run-canary-locally.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00121 | $0.03620 |
| Opus 5.5 | $0.00048 | $0.01448 |
| Sonnet 5.5 | $0.00024 | $0.00724 |
| Haiku 4.5 | $0.00012 | $0.00362 |
Grade C, and why
run-canary-locally scanned grade C with 1 finding against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 15d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Recursive force deletehighDestructive command
rm -rf with a variable or a broad path is one typo away from removing the wrong tree.
rm -rf /tmp/canary-run && mkdir -p /tmp/canary-run How it starts
The opening of the file, as written. The whole thing — 329 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Running the Canaries Locally
Repository: mitodl/ol-infrastructure.
Every unqualified path below is relative to that checkout.
When to use
- A
canary-<property>build went red and you want to reproduce it. - You are writing or changing a spec and need the edit/run loop.
- You want to read a failed run's trace.
For what to write and where it goes, use the add-canary-journey skill. This
skill is only about getting a run to happen on your machine.
The authoritative design notes live in
src/ol_concourse/pipelines/canaries/AGENTS.md
and README.md. This
skill does not restate them; it gets you to a passing run.
Prerequisites
- Node and
npmon PATH. There is nouv/Python involvement in a canary run — Python only renders the pipeline, never the tests. - Network reach to the target property. Every current target is publicly reachable, so no VPN.
All commands below run from the canary directory:
cd src/ol_concourse/pipelines/canaries
Step 1 — Install, including the part npm ci does not do
npm ci
npx playwright install chromium
Both lines are required. In Playwright 1.63 the playwright,
playwright-core and @playwright/test packages all have
hasInstallScript=false and ship no install.js — verified in
package-lock.json — so npm ci installs six packages and downloads no
browser at all. Skipping the second line gets you:
browserType.launch: Executable doesn't exist at
/home/<you>/.cache/ms-playwright/chromium_headless_shell-1243/...
which reads like a broken install and is not one.
This is also why the pipeline does not need that second line: browsers are baked
into the mcr.microsoft.com/playwright image at a revision keyed to the exact
Playwright version, which is the whole reason package.json's pin is the single
source of truth for the image tag.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 15d ago Changed 913a83c884ef
- 25d ago First seen · 329 lines · 121 tokens per session scan C 913de4a945c2
run-canary-locally is a skill published in the GitHub repository mitodl/agent-kit (5 stars, last pushed yesterday), licensed BSD-3-Clause. It adds 121 tokens to every session and 3,620 once invoked, about $0.0005 per session on Opus 5.5. A static security scan graded it C with 1 finding (recursive force delete). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-23.
Other skills, from other repositories
browse
Drive a real browser through Aside: open a page, read it, click through a flow, take screenshots, check console errors. (gstack).
cua-driver
Use Cua Driver for desktop or browser tasks that are awkward or unavailable through Bash/APIs, or when the user explicitly wants GUI interaction: app testing, visual bug reproduction, form filling, calendar entry, screenshots, and demo recording. Also covers Cua setup; not OpenAI Codex Computer Use or web research.
create-verification-skill
Create a repo-local verification skill and exhaustive feature map for driving a real app through its UI, CLI, or API. Use for "create a verification skill", "make a verify skill for this repo", or "document how agents can verify this app".
browser-qa
A browser-based quality check for deployed web pages and user flows. It uses browser automation to test rendering, navigation, forms, interactions, responsive behaviour, and accessibility-related issues.
webapp-testing
Toolkit for interacting with and testing local web applications using Playwright. Supports verifying frontend functionality, debugging UI behavior, capturing browser screenshots, and viewing browser logs.
test-electron-app
Drive the real running PostHog Electron app (live tRPC, workspace-server, real data) over CDP with agent-browser. Connect to the running app on port 9222, test desktop changes against a local Django stack, snapshot the accessibility tree, inspect network requests, and screenshot only when explicitly asked. Use when…