Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add sonichi/sutando --skill voice-agent-test-harnessgit clone --depth 1 https://github.com/sonichi/sutandoWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/sonichi/sutando/voice-agent-test-harness)<a href="https://agentmods.dev/skills/sonichi/sutando/voice-agent-test-harness"><img src="https://agentmods.dev/badge/skills/sonichi/sutando/voice-agent-test-harness/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/sonichi/sutando/voice-agent-test-harness"><img src="https://agentmods.dev/badge/skills/sonichi/sutando/voice-agent-test-harness.svg" alt="Reviewed on agentmods" width="80" height="20"></a>- NVIDIA SkillSpector pass
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00000 | $0.00977 |
| Opus 5.5 | $0.00000 | $0.00391 |
| Sonnet 5.5 | $0.00000 | $0.00195 |
| Haiku 4.5 | $0.00000 | $0.00098 |
Grade A, and why
voice-agent-test-harness scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 1mo ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 59 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Voice-Agent Test Harness
Drive a fixed suite of spoken tests against a voice agent ("subject") from a co-located machine ("prober"), measure response latency / clarity / accuracy, diff against baseline, and report to the owner over Telegram.
Design: docs/voice-agent-test-framework.md
v1 (macOS). Real audio path: TTS via
gemini-tts+afplay, mic capture viasoxrec(CoreAudio), voice-onset via numpy RMS, STT + judge via Gemini (Sutando-standard,GEMINI_API_KEY). Manual trigger; reports to owner only. Each prober-side component is tested; the full closed loop needs the second laptop speaking.
What runs end-to-end today
So that a half-SKIPPED suite is never mistaken for "mostly fine," here is exactly what executes through the real acoustic path now versus what is stubbed or excluded. A captured live run is committed at examples/run-2026-06-06.json.
| Capability | Status today |
|---|---|
Single-answer suite (test_cases.yaml, core-v1) — speak → capture → onset → Gemini STT → judge → score |
✅ Wired. Every row runs end-to-end on real audio; pass / fail / partial / no_response are all measured outcomes, not stubs. |
| Latency / clarity / accuracy scoring + baseline diff + Telegram roll-up | ✅ Wired — computed on real captured turns. |
timer action test — real side-effect verify (waits, listens for the alarm) |
✅ Wired. |
Multi-turn workflow turns (workflow_cases.yaml, e.g. the developer code-change flow) |
⚠️ Partial. The spoken handling is captured and judged; remote side effects (branch/test/cleanup) are not observable from the prober, so these score wording only. |
| Gmail / CRM workflow turns | ⛔ Excluded — unfinished test setup; omitted from results, not reported as failures. |
| Daily auto-scheduling | ⛔ Not wired — manual trigger only. |
How to try it (two laptops, same room)
- Subject: on laptop 2, start a normal Sutando voice session, mic open, speaker up.
- Prober: on laptop 1 (this one), grant Terminal Microphone permission (System Settings → Privacy → Microphone), then:
cd ~/GitHub/sutando/skills/voice-agent-test-harness python3 scripts/run_suite.py --quick # --quick shortens the 2-min timer wait to 30s - The prober speaks each prompt; the subject replies; the prober measures, transcribes, judges, and prints the roll-up. Add
--deliverto send the report to your Telegram.
What ships with it
12 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
- .gitignore 152 B
- examples/README.md 1.6 KB
- examples/run-2026-06-06.json 9.2 KB
- manifest.json 471 B
- results/.gitkeep 0 B
- scripts/audio.py 11 KB runs code
- scripts/baseline.py 3.4 KB runs code
- scripts/report.py 3.8 KB runs code
- scripts/run_suite.py 12 KB runs code
- scripts/score.py 6.9 KB runs code
- test_cases.yaml 8.4 KB
- workflow_cases.yaml 4.2 KB
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 1mo ago First seen · 59 lines · 0 tokens per session scan A 5ff084a6aed9
voice-agent-test-harness is a skill published in the GitHub repository sonichi/sutando (400 stars, last pushed today), licensed MIT. It costs nothing until one of its globs matches a file; then it loads 977 tokens. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-03.
Other skills, from other repositories
typescript-eval
Test TypeScript code snippets before persisting as skills.
xcode
Build, test and run iOS apps with xcodebuild: discover projects, list schemes, read the bundle id, parsed build errors (file:line), per-conversation defaults, and build-install-launch on the Simulator in one call.
mobile-simulators
Drive iOS simulators and Android emulators or devices with one vocabulary — boot, install, launch with logs, read the accessibility tree, tap/swipe/type by ref with proof on every touch, screenshots, video, push/location/permissions — under a driving indicator the user sees on the device window.
adk-go-self-review
Review an ADK Go change the way a maintainer will — a fresh-context pass over the whole diff, five lenses (correctness and tests, scope, simplicity, style, adk-python parity), and the mutation check that proves your tests pin the change. Use before opening a PR, before any later push that changes code, and when asked…
app-builder
Build and edit small, personal visual tools and artifacts — dashboards, trackers, calculators, data visualizations, charts, simple landing pages, and slide decks the user wants for THEMSELVES. This is the right skill whenever the user asks to "visualize this," "make a chart," or "build an artifact" for their own use…
gmail
Manage Gmail email — drafting, sending, organizing, filters, vacation replies, and inbox analysis.