What is test-opencode-tooling?

A testing workflow for checking whether OpenCode can reliably call tools through AFM. AFM is the system under test; the workflow captures streaming tool-call errors and classifies them as translation problems or model-generated errors.

Use it after changing tool-call parsing, when onboarding a model, or when investigating OpenCode tool failures. It can test prompts and selected models with different AFM startup options.

It helps identify whether a failed tool call comes from the AFM integration or from the model producing an invalid call. It produces a diagnostic report without changing the underlying code.

test-opencode-tooling — Skill by scouzi1966

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods

npx agentmods add skills/scouzi1966/maclocal-api/test-opencode-tooling

Any agent

npx skills add scouzi1966/maclocal-api --skill test-opencode-tooling

Clone the repo

git clone --depth 1 https://github.com/scouzi1966/maclocal-api

Made for: Claude Code, Codex.

Per session 47 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.

When invoked 4,414 The whole file, excluding the scripts and references it only reads on demand.

Security scan C 2 findings. Scan, not verified.

Origin original No closer match found in the catalogue.

Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

Model	Per session	Once invoked
Fable 5	$0.00047	$0.04414
Opus 5	$0.00023	$0.02207
Sonnet 5	$0.00009	$0.00883
Haiku 4.5	$0.00005	$0.00441

Measured 2d ago against content hash c111e9a4fe17, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade C, and why

test-opencode-tooling scanned grade C with 2 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Recursive force deletehighDestructive command

rm -rf with a variable or a broad path is one typo away from removing the wrong tree.

**IMPORTANT: Clean workdir between iterations.** Before each run, remove all generated files from the OpenCode workdir so that results from a previous iteration don't contaminate the next one (e.g., OpenCode's "must read

Makes network callslowCapability

Not a fault in itself. Listed so you know the mod talks to something, and to what.

until curl -sf http://127.0.0.1:${OC_PORT}/ >/dev/null 2>&1; do sleep 1; done

.claude/skills/test-opencode-tooling/SKILL.md · 402 lines

How it starts

The opening of the file, as written. The whole thing — 402 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Test OpenCode Tooling

Automated loop that runs OpenCode tasks against afm, captures tool call errors from both sides, classifies each as an afm bug or model error, and generates a report. Does not fix anything.

When to Use

After changing tool call parsing code (XML, streaming, type coercion)
Onboarding a new model to verify tool call reliability
Investigating user-reported tool call failures with OpenCode
Comparing tool call error rates across models

First Questions to Ask

Prompt/PRD — Ask the user to paste the prompt text or provide a file path. This is the task OpenCode will execute (e.g., a PRD, coding task, or test scenario that exercises tool calls).

Model(s) — Which model(s) to test? Show available:

MACAFM_MLX_MODEL_CACHE=/Volumes/edata/models/vesta-test-cache ./Scripts/list-models.sh

afm start parameters — Any extra flags beyond defaults? (e.g., --tool-call-parser afm_adaptive_xml, --enable-prefix-caching, --enable-grammar-constraints, --no-think). Recommended: --tool-call-parser afm_adaptive_xml --enable-grammar-constraints — this combination gives the highest tool call success rate (100% on 35B-A3B vs 60% without grammar constraints on realistic workloads).
Iterations — How many times to run the same prompt per model? Default: 1. More runs help distinguish flaky model errors from deterministic afm bugs.
Working directory — Temp dir for OpenCode to work in. Default: create a fresh /tmp/opencode-test-TIMESTAMP per run.

OpenCode CLI Gotchas

CRITICAL: opencode run hangs silently without a PTY. It prints one INFO line and freezes — no error, no output. You must use one of these approaches:

opencode serve + run --attach (recommended): Start a headless server, then attach run to it via expect for PTY
expect wrapper: Provides the pseudo-TTY that opencode run requires

Other gotchas:

opencode.json model field must be a string, not an object — "model": "ollama/model-id" not "model": {"default": "..."}
The npm provider format (@ai-sdk/openai-compatible) is required for custom baseURL — the "api": "openai" format does NOT accept baseURL
OpenCode config is loaded from both ~/.config/opencode/opencode.json (global) AND $WORKDIR/opencode.json (local) — local overrides global
The workdir should be a git repo (git init) for OpenCode to function properly

Read the full file on GitHub · 402 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

2d ago First seen · 402 lines · 47 tokens per session scan C c111e9a4fe17

Subscribe to this mod's changes

test-opencode-tooling is a skill published in the GitHub repository scouzi1966/maclocal-api (334 stars, last pushed 2d ago), licensed MIT. It adds 47 tokens to every session and 4,414 once invoked, about $0.0002 per session on Opus 5. A static security scan graded it C with 2 findings (recursive force delete, makes network calls). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Debugging Localization Testing ai apple-foundation-models apple-intelligence local-llm macos-app macos-swift

heygen

Prepare guarded HeyGen Direct API requests for avatar video generation, video translation, asset discovery, and status polling.

HybridAIOne/hybridclaw · 25 tokens

capture-usage4claude-screenshots

Automate Usage4Claude interface screenshots with CleanShot X on macOS. Use when Codex needs to capture localized Usage4Claude menu-bar popover screenshots, switch Usage4Claude display/language settings, save files such as [email protected], or troubleshoot CleanShot/window-capture automation that depends on…

f-is-h/Usage4Claude · 97 tokens

test-opencode-tooling