grackle: Skill for Claude Code

.claude/skills/test-copilot-runtime/SKILL.md

test-copilot-runtime is a skill for Claude Code from nick-pape/grackle. It costs 59 tokens per session (709 once invoked), scanned A, original, MIT.

A testing workflow for running GitHub Copilot's coding-agent runtime against a test server. It documents which model names work, including that Claude Sonnet 4.5 is confirmed and GPT-4o is unavailable.

In plain words
What is it for?
Use it with a test server to create a Copilot test agent, run commands through it, and verify runtime behavior.
Why use it?
It provides a repeatable way to exercise the Copilot runtime and avoids choosing a model that the command-line runtime cannot use.

Skill for Claude Code

Written for Claude Code: installed under .claude/. Also seen: mentions Codex.

This is nick-pape/grackle's own configuration. It tells Claude Code how to work on grackle itself, so it is not a mod to install elsewhere. Copy it as a starting point and replace the rules that are about this project. Everything grackle configures →

Reuse

Borrowing it

Nothing to install: this file belongs to nick-pape/grackle. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.

Copy the file
curl -O https://raw.githubusercontent.com/nick-pape/grackle/main/.claude/skills/test-copilot-runtime/SKILL.md
Clone the repo
git clone --depth 1 https://github.com/nick-pape/grackle

Made for: Claude Code.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for test-copilot-runtime

README.md
[![agentmods](https://agentmods.dev/badge/skills/nick-pape/grackle/test-copilot-runtime/github.svg)](https://agentmods.dev/skills/nick-pape/grackle/test-copilot-runtime)
Your own site
<a href="https://agentmods.dev/skills/nick-pape/grackle/test-copilot-runtime"><img src="https://agentmods.dev/badge/skills/nick-pape/grackle/test-copilot-runtime/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

agentmods 80×15 button for test-copilot-runtime

Your own site · 80×15
<a href="https://agentmods.dev/skills/nick-pape/grackle/test-copilot-runtime"><img src="https://agentmods.dev/badge/skills/nick-pape/grackle/test-copilot-runtime.svg" alt="Reviewed on agentmods" width="80" height="20"></a>
Per session 59 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 709 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00059 $0.00709
Opus 5.5 $0.00024 $0.00284
Sonnet 5.5 $0.00012 $0.00142
Haiku 4.5 $0.00006 $0.00071

Measured 2mo ago against content hash 587b3e7d140b, method: parsed. Prices are Anthropic first-party input rates as of 2026-10-07, from the pricing page.

Security

Grade A, and why

test-copilot-runtime scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2mo ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

.claude/skills/test-copilot-runtime/SKILL.md · 47 lines

How it starts

The opening of the file, as written. The whole thing — 47 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Test the GitHub Copilot runtime

How to spawn the copilot runtime against an isolated test server. Assumes a server from /launch-grackle (with GRACKLE_URL + GRACKLE_API_KEY exported).

Allowed model names ✅ / ❌

Model Works? Notes
claude-sonnet-4.5 ✅ confirmed live The @github/copilot-sdk README default; spawns and runs
gpt-5 ⚠️ likely Documented for Copilot CLI; not individually confirmed here
claude-sonnet-4 ⚠️ likely From GitHub's supported-models docs
claude-haiku-4.5 ⚠️ likely From GitHub's supported-models docs
gpt-4o ❌ does NOT work Model "gpt-4o" is not available. GPT-4o is intentionally absent from the Copilot CLI/SDK (only in Copilot Chat + the VS Code extension)

⚠️ grackle runtimes advertises gpt-4o for copilot — that is wrong/unavailable for the CLI. Use claude-sonnet-4.5. (Catalog entry: packages/common/src/runtime-catalog.ts; worth fixing — see the runtime-catalog.)

Spawn it

grackle persona create "Copilot Tester" --runtime copilot --model claude-sonnet-4.5 \
  --prompt "You are a test agent. When asked to run a command, use your shell tool to run exactly that one command and nothing else."
grackle spawn local "Use your shell tool to run exactly this one command, then stop: cat /nonexistent_file_xyz" --persona copilot-tester

Read the full file on GitHub · 47 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 2mo ago First seen · 47 lines · 0 tokens per session scan A 587b3e7d140b

Subscribe to this mod's changes

test-copilot-runtime is a skill published in the GitHub repository nick-pape/grackle (22 stars, last pushed 4mo ago), licensed MIT. It adds 59 tokens to every session and 709 once invoked, about $0.0002 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Related

Other skills, from other repositories

rpa-init

Run when the user invokes /rpa-init or asks to onboard or warm up context on a repository. The agent studies code, reads documentation and test code, sets up the dev environment as the project expects, runs tests, and writes a short project report. No extra user brief is required.

coddy-project/coddy-agent · 63 tokens

rpa-gen-rules

Use when the user invokes /rpa-gen-rules or asks to create, refresh, audit, or synchronize project instructions such as root AGENTS.md, Cursor .cursor/rules, or Claude Code .claude/rules, especially when existing rule trees have drifted.

coddy-project/coddy-agent · 61 tokens

rpa-feat

Run when the user invokes /rpa-feat together with a clear feature description (for example issue text). BDD workflow: plan, failing tests, implementation, green tests, full suite, docs and examples, linter at the end. Will not work correctly without the user supplying what to build.

coddy-project/coddy-agent · 66 tokens

rpa-bugfix

Run when the user invokes /rpa-bugfix together with a clear bug description (expected vs actual, reproduction). Reproduction test first, then fix, full test suite, short report. Will not work correctly without the bug details.

coddy-project/coddy-agent · 54 tokens

configure-coddy

Change Coddy's own configuration when the user asks for it: edit settings, providers, models, logging, permissions, or find, install, update, and remove MCP servers and skills. Stages UCI-style commands for config.yaml and commits only after the user confirms saving; MCP servers are entries of the mcp.json files. Load…

coddy-project/coddy-agent · 120 tokens

code-guidelines-go

Go 1.24–1.27 coding guidelines for the dimetron/pi-go AI agent runtime. Use this skill whenever writing, reviewing, or refactoring ANY Go code in pi-go. This covers idiomatic style, error handling, concurrency, project layout, testing (table-driven, fuzz, benchmarks, synctest), new stdlib usage, golangci-lint v2…

dimetron/pi-go · 124 tokens