What is test-acp-runtime?

A test workflow for starting and exercising ACP agent runtimes against an isolated Grackle test server. ACP is a standard protocol for connecting coding agents to applications, and these runtimes include Claude Code, Codex, and Copilot variants.

Use it to test Claude Code ACP, Codex ACP, or Copilot ACP against a test server and confirm their tool calls and model configuration work.

It helps verify how each runtime behaves and exposes real tool failures that some native runtimes may hide. It also documents the model settings needed to start each runtime.

How nick-pape/grackle configures Claude Code

Reuse

Borrowing it

Nothing to install: this file belongs to nick-pape/grackle. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.

Copy the file

curl -O https://raw.githubusercontent.com/nick-pape/grackle/main/.claude/skills/test-acp-runtime/SKILL.md

Clone the repo

git clone --depth 1 https://github.com/nick-pape/grackle

Made for: Claude Code.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

README.md

[![agentmods](https://agentmods.dev/badge/skills/nick-pape/grackle/test-acp-runtime/github.svg)](https://agentmods.dev/skills/nick-pape/grackle/test-acp-runtime)

Your own site

<a href="https://agentmods.dev/skills/nick-pape/grackle/test-acp-runtime"><img src="https://agentmods.dev/badge/skills/nick-pape/grackle/test-acp-runtime/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

Your own site · 80×15

<a href="https://agentmods.dev/skills/nick-pape/grackle/test-acp-runtime"><img src="https://agentmods.dev/badge/skills/nick-pape/grackle/test-acp-runtime.svg" alt="Reviewed on agentmods" width="80" height="20"></a>

Per session 64 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.

When invoked 784 The whole file, excluding the scripts and references it only reads on demand.

Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe.

Origin original No closer match found in the catalogue.

Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

Model	Per session	Once invoked
Fable 5.1	$0.00064	$0.00784
Opus 5.5	$0.00026	$0.00314
Sonnet 5.5	$0.00013	$0.00157
Haiku 4.5	$0.00006	$0.00078

Measured 1mo ago against content hash 737b7fb8cf28, method: parsed. Prices are Anthropic first-party input rates as of 2026-10-07, from the pricing page.

Security

Grade A, and why

test-acp-runtime scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 1mo ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

.claude/skills/test-acp-runtime/SKILL.md · 46 lines

How it starts

The opening of the file, as written. The whole thing — 46 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Test the ACP runtimes

How to spawn the ACP-variant runtimes — claude-code-acp, codex-acp, copilot-acp — against an isolated test server. Assumes a server from /launch-grackle (with GRACKLE_URL + GRACKLE_API_KEY exported).

Why ACP matters: the native runtimes (claude-code/copilot/codex) are slated to be deprecated in favor of these ACP variants. ACP is a standard protocol with a uniform tool_call_update status (completed/failed), so one adapter (runtime-acp) covers all agents — and unlike native claude-code (which emits synthetic empty tool_results and can't surface failures), claude-code-acp reports real tool failures.

Model names

ACP runtimes are listed as model "(agent-selected)" by grackle runtimes, but a persona still requires a model (spawn errors with FailedPrecondition: persona has no model configured otherwise):

Runtime	Model to pass
`claude-code-acp`	`sonnet` / `opus` / `haiku` (runs Claude underneath) — validated with `sonnet`
`codex-acp`	a Codex model (same ChatGPT-account gating as native — use `gpt-5.5`; see /test-codex-runtime)
`copilot-acp`	a Copilot model, e.g. `claude-sonnet-4.5` (see /test-copilot-runtime)

Spawn it

grackle persona create "ACP Tester" --runtime claude-code-acp --model sonnet --prompt "You are a test agent."
grackle spawn local "Run exactly this one shell command and nothing else, then stop: cat /nonexistent_file_xyz" --persona acp-tester

What it proves (confirmed live 2026-05-29)

Spawns cleanly. The earlier Invalid permissions.defaultMode: auto failure (#1366) is fixed by #1370 — claude-code-acp now isolates its CLAUDE_CONFIG_DIR from your personal ~/.claude (which may carry the interactive-only defaultMode: "auto").
Surfaces real tool failures. A failing command (cat /nonexistent) produced a tool_result with content {is_ok:false} + "tool_error":true and the real Exit code 1 / No such file text — via the adapter's status === "failed" → toolError mapping (packages/runtime-acp/src/acp.ts). This is the case native claude-code cannot express.

Read the full file on GitHub · 46 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

1mo ago First seen · 46 lines · 64 tokens per session scan A 737b7fb8cf28

Subscribe to this mod's changes

test-acp-runtime is a skill published in the GitHub repository nick-pape/grackle (22 stars, last pushed 4mo ago), licensed MIT. It adds 64 tokens to every session and 784 once invoked, about $0.0003 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Building agents Claude Code Testing acp agent-client-protocol agentic-workflow ai-agents cli

rpa-init

Run when the user invokes /rpa-init or asks to onboard or warm up context on a repository. The agent studies code, reads documentation and test code, sets up the dev environment as the project expects, runs tests, and writes a short project report. No extra user brief is required.

coddy-project/coddy-agent · 63 tokens

rpa-gen-rules

Use when the user invokes /rpa-gen-rules or asks to create, refresh, audit, or synchronize project instructions such as root AGENTS.md, Cursor .cursor/rules, or Claude Code .claude/rules, especially when existing rule trees have drifted.

coddy-project/coddy-agent · 61 tokens

rpa-feat

Run when the user invokes /rpa-feat together with a clear feature description (for example issue text). BDD workflow: plan, failing tests, implementation, green tests, full suite, docs and examples, linter at the end. Will not work correctly without the user supplying what to build.

coddy-project/coddy-agent · 66 tokens

rpa-bugfix

Run when the user invokes /rpa-bugfix together with a clear bug description (expected vs actual, reproduction). Reproduction test first, then fix, full test suite, short report. Will not work correctly without the bug details.

coddy-project/coddy-agent · 54 tokens

code-guidelines-go

Go 1.24–1.27 coding guidelines for the dimetron/pi-go AI agent runtime. Use this skill whenever writing, reviewing, or refactoring ANY Go code in pi-go. This covers idiomatic style, error handling, concurrency, project layout, testing (table-driven, fuzz, benchmarks, synctest), new stdlib usage, golangci-lint v2…

dimetron/pi-go · 124 tokens

go-127

What changed in Go 1.27 (released August 2026) and how it changes the way Go is written in pi-go. Use this skill when writing or reviewing Go that could use a 1.27 feature, when bumping the go directive in go.mod, when a build or test behaves differently after a toolchain upgrade, or when code-guidelines-go points…

dimetron/pi-go · 177 tokens

grackle: Skill for Claude Code