Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add sonichi/sutando --skill voice-agent-test-harnessgit clone --depth 1 https://github.com/sonichi/sutandoWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/sonichi/sutando/voice-agent-test-harness)<a href="https://agentmods.dev/skills/sonichi/sutando/voice-agent-test-harness"><img src="https://agentmods.dev/badge/skills/sonichi/sutando/voice-agent-test-harness.svg" alt="Measured on agentmods" height="20"></a>- NVIDIA SkillSpector pass
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00000 | $0.00977 |
| Opus 5 | $0.00000 | $0.00489 |
| Sonnet 5 | $0.00000 | $0.00195 |
| Haiku 4.5 | $0.00000 | $0.00098 |
Grade A, and why
voice-agent-test-harness scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 5d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 59 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Voice-Agent Test Harness
Drive a fixed suite of spoken tests against a voice agent ("subject") from a co-located machine ("prober"), measure response latency / clarity / accuracy, diff against baseline, and report to the owner over Telegram.
Design: docs/voice-agent-test-framework.md
v1 (macOS). Real audio path: TTS via
gemini-tts+afplay, mic capture viasoxrec(CoreAudio), voice-onset via numpy RMS, STT + judge via Gemini (Sutando-standard,GEMINI_API_KEY). Manual trigger; reports to owner only. Each prober-side component is tested; the full closed loop needs the second laptop speaking.
What runs end-to-end today
So that a half-SKIPPED suite is never mistaken for "mostly fine," here is exactly what executes through the real acoustic path now versus what is stubbed or excluded. A captured live run is committed at examples/run-2026-06-06.json.
| Capability | Status today |
|---|---|
Single-answer suite (test_cases.yaml, core-v1) — speak → capture → onset → Gemini STT → judge → score |
✅ Wired. Every row runs end-to-end on real audio; pass / fail / partial / no_response are all measured outcomes, not stubs. |
| Latency / clarity / accuracy scoring + baseline diff + Telegram roll-up | ✅ Wired — computed on real captured turns. |
timer action test — real side-effect verify (waits, listens for the alarm) |
✅ Wired. |
Multi-turn workflow turns (workflow_cases.yaml, e.g. the developer code-change flow) |
⚠️ Partial. The spoken handling is captured and judged; remote side effects (branch/test/cleanup) are not observable from the prober, so these score wording only. |
| Gmail / CRM workflow turns | ⛔ Excluded — unfinished test setup; omitted from results, not reported as failures. |
| Daily auto-scheduling | ⛔ Not wired — manual trigger only. |
How to try it (two laptops, same room)
- Subject: on laptop 2, start a normal Sutando voice session, mic open, speaker up.
- Prober: on laptop 1 (this one), grant Terminal Microphone permission (System Settings → Privacy → Microphone), then:
cd ~/GitHub/sutando/skills/voice-agent-test-harness python3 scripts/run_suite.py --quick # --quick shortens the 2-min timer wait to 30s - The prober speaks each prompt; the subject replies; the prober measures, transcribes, judges, and prints the roll-up. Add
--deliverto send the report to your Telegram.
What ships with it
12 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
- .gitignore 152 B
- examples/README.md 1.6 KB
- examples/run-2026-06-06.json 9.2 KB
- manifest.json 471 B
- results/.gitkeep 0 B
- scripts/audio.py 11 KB runs code
- scripts/baseline.py 3.4 KB runs code
- scripts/report.py 3.8 KB runs code
- scripts/run_suite.py 12 KB runs code
- scripts/score.py 6.9 KB runs code
- test_cases.yaml 8.4 KB
- workflow_cases.yaml 4.2 KB
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 5d ago First seen · 59 lines · 0 tokens per session scan A 5ff084a6aed9
voice-agent-test-harness is a skill published in the GitHub repository sonichi/sutando (394 stars, last pushed today), licensed MIT. It costs nothing until one of its globs matches a file; then it loads 977 tokens. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-03.
Other skills, from other repositories
typescript-eval
Test TypeScript code snippets before persisting as skills.
ccg-workflow
How to run a non-trivial change end to end with the CCG role tools (ccganalyze / ccgdesign / ccgbuild / ccgdebug / ccgoptimize / ccgreview / ccgtest) and the verify- quality gates. Use when a task is a whole feature, a refactor, a bug whose cause is not yet known, or any change large enough that one straight-through…
app-builder
Build and edit small, personal visual tools and artifacts — dashboards, trackers, calculators, data visualizations, charts, simple landing pages, and slide decks the user wants for THEMSELVES. This is the right skill whenever the user asks to "visualize this," "make a chart," or "build an artifact" for their own use…
assistant-migration
Migrate from ChatGPT, Claude, OpenClaw, Hermes, Manus, and other AI assistants into Vellum by inspecting their data exports, conversation archives, files, prompts, custom instructions, memory, saved memories, tools, GPTs, workflows, integrations, and relationships, then mapping as much as safely possible into Vellum…
vellum-memory-v3-migration
One-time migration of an existing memory-v2 concept corpus into the memory-v3 section-grain "wiki" — topical articles with a stand-alone lead and queryable sections — with loss-proof staging, assistant-reviewed authoring, and a retrieval-eval gate before cutover.
media-processing
Ingest and process media files (video, audio, image).