Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add skills/scouzi1966/maclocal-api/test-afm-assertionsnpx skills add scouzi1966/maclocal-api --skill test-afm-assertionsgit clone --depth 1 https://github.com/scouzi1966/maclocal-apiWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/scouzi1966/maclocal-api/test-afm-assertions)<a href="https://agentmods.dev/skills/scouzi1966/maclocal-api/test-afm-assertions"><img src="https://agentmods.dev/badge/skills/scouzi1966/maclocal-api/test-afm-assertions.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5 | $0.00000 | $0.02549 |
| Opus 5 | $0.00000 | $0.01274 |
| Sonnet 5 | $0.00000 | $0.00510 |
| Haiku 4.5 | $0.00000 | $0.00255 |
Grade A, and why
test-afm-assertions scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 4d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 188 lines — stays where its author put it; the contents beside it link to each section on GitHub.
test-afm-assertions
Run AFM assertion tests across one or more models with deterministic pass/fail validation. Tests tool calling (including XML format), stop sequences, logprobs, think extraction, streaming, prompt cache, error handling, and performance.
Triggers
Use this skill when the user asks to:
- Run assertion tests against specific models
- Validate tool calling across Qwen3/Qwen3.5 models
- Compare models on the same assertion suite
- Regression test after code changes with multiple models
- Validate XML tool call parsing for Qwen3-Coder or Qwen3.5 models
First Questions to Ask
- Which model(s)? — Ask the user which model(s) to test. Show available models:
MACAFM_MLX_MODEL_CACHE=/Volumes/edata/models/vesta-test-cache ./Scripts/list-models.sh - Tier? — unit (offline, ~5s), smoke (~2 min/model), standard (~5 min/model), or full (~15 min/model)?
Default: standard. Use
unitfor offline-only Swift tests (no server needed). - Forced parser? — Should we also test with
--tool-call-parser qwen3_xml? Suggest yes if testing Qwen3/Qwen3.5 models. - Grammar constraints? — Should we test with
--enable-grammar-constraints? Recommended yes — EBNF grammar-constrained decoding forces valid XML tool call structure at generation time, preventing JSON-inside-XML format switching and missing required parameters. Requires--tool-call-parser afm_adaptive_xml. Matrix testing shows grammar constraints improve tool call success from 60% to 100% on realistic workloads (35B-A3B model). - Server already running? — If yes, use single-model mode (
test-assertions.sh). If no (or multiple models), use multi-model mode (test-assertions-multi.sh).
Common Model Configurations
Qwen3 XML Format Models (auto-detect xmlFunction)
| Model | Type | Size | Notes |
|---|---|---|---|
mlx-community/Qwen3.5-35B-A3B-4bit |
qwen3_5_moe | 19 GB | Primary test model |
mlx-community/Qwen3.5-9B-MLX-4bit |
qwen3_5 (dense) | 5.6 GB | Fast, dense Qwen3.5 |
mlx-community/Qwen3-Coder-Next-4bit |
qwen3_5_moe | 42 GB | Coder variant, no thinking |
mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bit |
qwen3_moe | 16 GB | Coder MoE |
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 4d ago First seen · 188 lines · 0 tokens per session scan A b5caaee2765f
test-afm-assertions is a skill published in the GitHub repository scouzi1966/maclocal-api (337 stars, last pushed today), licensed MIT. It costs nothing until one of its globs matches a file; then it loads 2,549 tokens. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
apple-foundation-models-app-patterns
Apply MVVM architecture patterns for building Foundation Models apps.
apple-foundation-models-conversation-context-builder
Extract context and build continuation prompts for multi-turn conversations.
apple-foundation-models-custom-tools
Build custom tools that the model can call to extend functionality.
apple-foundation-models-dynamic-schemas
Build schemas dynamically at runtime for flexible structured output.
apple-foundation-models-error-recovery
Handle context overflow, refusals, rate limits, and generation errors.
apple-foundation-models-feedback-sentiment-api
Collect user feedback and analyze sentiment to improve model responses.