agento11y-test-starter

agento11y-test-starter is a skill for Claude Code from grafana/gcx. It costs 248 tokens per session (7,053 once invoked), scanned A, original, Apache-2.0.

A starter testing workflow for an AI agent before it is released or receives real users. It reads the agent's instructions, tools, and task, then creates test cases and a local test runner when possible.

In plain words
What is it for?
Use it to create and review an offline test suite, recommend ways to score an agent, and prepare a runner connected to the agent's software development kit when the agent can be called easily.
Why use it?
It helps solve the problem of deciding what to test and how to judge the answers before real usage provides examples. The tests include normal, unusual, and deliberately difficult requests.

Skill for Claude Code ✓ vendor

Written for Claude Code: shipped in a Claude Code plugin.

Needs its repository: it runs a file that does not travel with it, so clone the repository first. The line is python evals/run_experiment.py.

Part of the gcx plugin — 24 skills, 1 agent shipped together

Good fit Use it to create and review an offline test suite, recommend ways to score an agent, and prepare a runner connected to the agent's software development kit when the agent can be called easily.

Compare 6 skills from other repositories ↓
About the project

gcx is a command-line tool that lets people and AI coding agents manage and inspect Grafana Cloud, Enterprise, and open-source instances. It provides access to dashboards, alerts, SLOs, metrics, logs, and traces, with workflows for alert investigation, dashboard management, GitOps, and observability setup. Its catalogue entries provide agent instructions and extensions for using gcx.

grafana/gcx · 594 stars · on GitHub · grafana.com

Install

Getting it into your agent

It runs from inside its repository, so the clone comes first — what it calls does not travel with the file alone.

Clone the repo
git clone --depth 1 https://github.com/grafana/gcx
agentmods
npx agentmods add skills/grafana/gcx/agento11y-test-starter

Made for: Claude Code.

Or install gcx, the plugin that ships this one along with the rest of its 24 skills, 1 agent.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for agento11y-test-starter

README.md
[![agentmods](https://agentmods.dev/badge/skills/grafana/gcx/agento11y-test-starter/github.svg)](https://agentmods.dev/skills/grafana/gcx/agento11y-test-starter)
Your own site
<a href="https://agentmods.dev/skills/grafana/gcx/agento11y-test-starter"><img src="https://agentmods.dev/badge/skills/grafana/gcx/agento11y-test-starter/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

agentmods 80×15 button for agento11y-test-starter

Your own site · 80×15
<a href="https://agentmods.dev/skills/grafana/gcx/agento11y-test-starter"><img src="https://agentmods.dev/badge/skills/grafana/gcx/agento11y-test-starter.svg" alt="Reviewed on agentmods" width="80" height="20"></a>
Per session 248 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 7,053 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe. Third-party audits
  • NVIDIA SkillSpector warn 7 Sept 2026
SkillSpector: 1 finding, up to medium

These are SkillSpector’s own severities. On a checked sample its high-severity flags on skills were ~96% false positives — a documented command, a public API, a “never do X” rule — so we show them as a caution to read, not a verdict. Why →

  • medium Excessive Agency · line 70
    Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.
    Fix: Add human-in-the-loop confirmation for destructive, irreversible, or high-impact operations. Never auto-execute commands that modify files, send data, or alter system state.
How audits are shown
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00248 $0.07053
Opus 5 $0.00124 $0.03527
Sonnet 5 $0.00050 $0.01411
Haiku 4.5 $0.00025 $0.00705

Measured 10d ago against content hash d703e4e3574a, method: parsed. Prices are Anthropic first-party input rates as of 2026-09-10, from the pricing page.

Security

Grade A, and why

agento11y-test-starter scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 10d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

claude-plugin/skills/agento11y-test-starter/SKILL.md · 477 lines

How it starts

The opening of the file, as written. The whole thing — 477 lines — stays where its author put it; the contents beside it link to each section on GitHub.

agento11y test starter

Help a developer test an AI agent before it ships — while there is no real traffic yet. The hard part isn't running the test — it's having cases to test against and knowing how to score them, grounded in the agent's actual code.

Scope: this is the pre-production, offline skill — it writes test cases and a local runner, and never touches the tenant. Once the agent is deployed and has real traffic, setting up online eval rules + guards on that traffic is a different skill, agento11y-prod-setup.

Always produce:

  1. A ranked, justified evaluator recommendation for this agent.
  2. A starter suite YAML the developer reviews and extends.

Then, depending on how runnable the agent is (Step 1):

  1. For an easily-invoked agent, a runner stub (run_experiment.py) that wires the suite to the SDK with two holes to fill — and optionally run it (Step 6), only with permission. For an agent that needs a harness or full runtime, point to the existing eval infra instead of a runner that can't actually call it.

This skill is language-agnostic — the reading, recommending, and YAML it produces do not depend on the agent's language. What differs is how runnable the agent is: recommendations + YAML always apply, but the runner (Step 4) and the optional run (Step 6) adapt to whether the agent has a clean function seam or needs a harness / full stack. For deeper run-side patterns (binding existing generations, cross-process verifiers) point to the per-language run skill (Python: agento11y-experiments).

Note: agento11y-experiments currently ships in the grafana/agento11y repo (python/skills/agento11y-experiments/), not in this gcx bundle yet — install it from there for now. Consolidating it into the gcx bundle is pending.

Prerequisites

The generated runner imports the Agent Observability SDK. Install it in the agent's environment before running (Step 6):

  • Python: pip install agento11y python-dotenv (the experiments API lives in agento11y.experiments; the runner uses python-dotenv to load the agent's .env, and it is not a dependency of agento11y)
  • Go: add github.com/grafana/agento11y/go

Read the full file on GitHub · 477 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 10d ago First seen · 477 lines · 248 tokens per session scan A d703e4e3574a

Subscribe to this mod's changes

agento11y-test-starter is a skill published in the GitHub repository grafana/gcx (594 stars, last pushed today), licensed Apache-2.0. It adds 248 tokens to every session and 7,053 once invoked, about $0.0012 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Related

Other skills, from other repositories

research-engineer

An uncompromising Academic Research Engineer. Operates with absolute scientific rigor, objective criticism, and zero flair. Focuses on theoretical correctness, formal verification, and optimal implementation across any required technology.

davila7/claude-code-templates · 43 tokens

tika-eval-compare

Compare extracts from two Tika builds over a corpus to detect regressions in content, encoding, exceptions, and embedded-document handling. Use for "compare before/after extracts", "eval this change against the corpus".

apache/tika · 50 tokens

neuron-evaluation-engineer

Create and run AI evaluations with datasets, assertions, and output drivers in Neuron AI. Use this skill whenever the user mentions evaluation, testing AI systems, creating evaluators, dataset-driven testing, assertion-based validation, or wants to measure AI system performance. Also trigger for tasks involving…

neuron-core/neuron-ai · 77 tokens

jetson-validate-image

Use after jetson-flash-image to run static BSP checks, on-target smoke/regression tests on a flashed DUT, or both. Not for build or flash steps. Triggers: validate bsp, on-target validation.

NVIDIA/skills · 50 tokens

atmos-validation

Validate Atmos projects, components, arbitrary JSON Schema inputs, EditorConfig, and GitHub Actions; use affected-file selection and native CI annotations.

cloudposse/atmos · 31 tokens

skill-benchmark

Benchmark AI skill effectiveness by measuring implementation quality against legacy constraints.

HoangNguyen0403/agent-skills-standard · 16 tokens