smoke-test

smoke-test is a skill for Claude Code, Codex from stacklok/toolhive-registry-server. It costs 40 tokens per session (8,653 once invoked), scanned B, original, Apache-2.0.

A smoke-test routine for the Registry Server, a service that lists and manages agent add-ons. It starts the service with Docker Compose and checks its HTTP endpoints with curl.

In plain words
What is it for?
Checking health, readiness, version, OpenAPI documentation, read-only and admin APIs, entry creation or updates, and OAuth or authentication enforcement.
Why use it?
It quickly reveals whether the server starts, responds, exposes its documented API, manages entries, and enforces authentication where required.

Skill for Claude CodeCodex

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add skills/stacklok/toolhive-registry-server/smoke-test
Any agent
npx skills add stacklok/toolhive-registry-server --skill smoke-test
Clone the repo
git clone --depth 1 https://github.com/stacklok/toolhive-registry-server

Made for: Claude Code, Codex.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for smoke-test

README.md
[![agentmods](https://agentmods.dev/badge/skills/stacklok/toolhive-registry-server/smoke-test.svg)](https://agentmods.dev/skills/stacklok/toolhive-registry-server/smoke-test)
Your own site
<a href="https://agentmods.dev/skills/stacklok/toolhive-registry-server/smoke-test"><img src="https://agentmods.dev/badge/skills/stacklok/toolhive-registry-server/smoke-test.svg" alt="Measured on agentmods" height="20"></a>
Per session 40 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 8,653 The whole file, excluding the scripts and references it only reads on demand.
Security scan B 2 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00040 $0.08653
Opus 5 $0.00020 $0.04326
Sonnet 5 $0.00008 $0.01731
Haiku 4.5 $0.00004 $0.00865

Measured 5d ago against content hash bd60fc8cb84c, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade B, and why

smoke-test scanned grade B with 2 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 5d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Sends data to an external URLmediumData exfiltration

A POST to an outside endpoint may be telemetry or may be exfiltration; either way the mod talks to somewhere, and you should know where.

curl_post() { curl -s -o /tmp/thv_body -w "%{http_code}" -X POST -H "Content-Type: application/json" -d "$2" "$BASE_URL$1"; }

Makes network callslowCapability

Not a fault in itself. Listed so you know the mod talks to something, and to what.

description: Start the Registry Server via docker compose and run a suite of curl-based smoke tests covering system, read-only MCP API, admin API, entry lifecycle, and OAuth/auth enforcement scenarios.
.claude/skills/smoke-test/SKILL.md · 753 lines

How it starts

The opening of the file, as written. The whole thing — 753 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Smoke Tests — Registry Server

Starts the Registry Server stack with docker compose, waits for it to be ready, then executes a suite of curl smoke tests organised as Gherkin scenarios. Pass keep-up as an argument to leave the stack running after the tests complete (useful for interactive exploration).

Embedded Scenarios

Feature: Registry Server smoke tests

  Background:
    Given the Registry Server is running at http://localhost:8080
    And the "default" registry is seeded from the upstream-registry.json file source

  # ── System endpoints ──────────────────────────────────────────────────────

  Scenario: Health check
    When I GET /health
    Then the response status is 200
    And the body contains "healthy"

  Scenario: Readiness check
    When I GET /readiness
    Then the response status is 200
    And the body contains "ready"

  Scenario: Version endpoint
    When I GET /version
    Then the response status is 200
    And the body contains "version"

  Scenario: OpenAPI spec
    When I GET /openapi.json
    Then the response status is 200
    And the body contains "openapi"

  # ── MCP Registry v0.1 — read-only ─────────────────────────────────────────

  Scenario: List servers in default registry
    When I GET /registry/default/v0.1/servers
    Then the response status is 200
    And the body contains "servers"

  Scenario: List servers with search filter
    When I GET /registry/default/v0.1/servers?search=mysql
    Then the response status is 200
    And the body contains "servers"

  Scenario: List servers with limit
    When I GET /registry/default/v0.1/servers?limit=2
    Then the response status is 200
    And the count of servers is at most 2

  Scenario: List servers filtered to latest version only
    When I GET /registry/default/v0.1/servers?version=latest
    Then the response status is 200
    And the body contains "servers"

  Scenario: List versions for a known server
    When I GET /registry/default/v0.1/servers/io.github.stacklok%2Fadb-mysql-mcp-server/versions
    Then the response status is 200
    And the body contains "servers"

  Scenario: Get a specific server version (latest)
    When I GET /registry/default/v0.1/servers/io.github.stacklok%2Fadb-mysql-mcp-server/versions/latest
    Then the response status is 200
    And the body contains "io.github.stacklok/adb-mysql-mcp-server"

  Scenario: Unknown registry returns 404
    When I GET /registry/nonexistent-registry/v0.1/servers
    Then the response status is 404

  Scenario: Unknown server version returns 404
    When I GET /registry/default/v0.1/servers/com.example%2Fdoes-not-exist/versions/1.0.0
    Then the response status is 404

  # ── Admin API v1 — registries ─────────────────────────────────────────────

  Scenario: List registries
    When I GET /v1/registries
    Then the response status is 200
    And the body contains "registries"

  Scenario: Get the default registry by name
    When I GET /v1/registries/default
    Then the response status is 200
    And the body contains "default"

  Scenario: Get a nonexistent registry returns 404
    When I GET /v1/registries/does-not-exist
    Then the response status is 404

  Scenario: Create a new registry via PUT
    When I PUT /v1/registries/test-registry with an empty source list
    Then the response status is 201
    And the body contains "test-registry"

  Scenario: Update an existing registry via PUT
    When I PUT /v1/registries/test-registry again with a description
    Then the response status is 200

  Scenario: Delete an API-created registry
    When I DELETE /v1/registries/test-registry
    Then the response status is 204

  Scenario: Delete a nonexistent registry returns 404
    When I DELETE /v1/registries/does-not-exist
    Then the response status is 404

  # ── Admin API v1 — sources ────────────────────────────────────────────────

  Scenario: List sources
    When I GET /v1/sources
    Then the response status is 200
    And the body contains "sources"

  Scenario: Get a known source by name
    When I GET /v1/sources/local-file
    Then the response status is 200
    And the body contains "local-file"

  Scenario: Get a nonexistent source returns 404
    When I GET /v1/sources/does-not-exist
    Then the response status is 404

  Scenario: Create a managed source
    When I PUT /v1/sources/managed-test with body {"managed":{}}
    Then the response status is 201
    And the body contains "managed-test"

  Scenario: Delete the managed source
    When I DELETE /v1/sources/managed-test
    Then the response status is 204

  # ── Entry lifecycle — publish and delete ──────────────────────────────────

  Scenario: Publish a server version
    Given a managed source exists
    When I POST /v1/entries with a server payload
    Then the response status is 201
    And the body contains the server name

  Scenario: Publish the same server version again returns 409
    When I POST /v1/entries with the same server payload
    Then the response status is 409

  Scenario: Publish a skill version
    When I POST /v1/entries with a skill payload
    Then the response status is 201
    And the body contains the skill name

  Scenario: Delete the published server version
    When I DELETE /v1/entries/server/com.example%2Ftest-server/versions/1.0.0
    Then the response status is 204

  Scenario: Delete a nonexistent entry returns 404
    When I DELETE /v1/entries/server/com.example%2Fnope/versions/9.9.9
    Then the response status is 404

  Scenario: Publish with both server and skill in body returns 400
    When I POST /v1/entries with both server and skill fields set
    Then the response status is 400

  Scenario: Publish with neither server nor skill in body returns 400
    When I POST /v1/entries with an empty payload
    Then the response status is 400

  # ── Claims update on edit ──────────────────────────────────────────────────

  Scenario: Registry claims are updated on PUT
    When I PUT /v1/registries/claims-test with claims {"org":"acme"}
    Then the response status is 201
    And the body contains "acme"
    When I PUT /v1/registries/claims-test with claims {"org":"contoso"}
    Then the response status is 200
    And the body contains "contoso"
    When I GET /v1/registries/claims-test
    Then the body contains "contoso"
    And the body does not contain "acme"

  Scenario: Source claims are updated on PUT
    When I PUT /v1/sources/claims-test with file-data and claims {"org":"acme"}
    Then the response status is 201
    When I GET /v1/sources/claims-test
    Then the body contains "acme"
    When I PUT /v1/sources/claims-test with file-data and claims {"org":"contoso"}
    Then the response status is 200
    When I GET /v1/sources/claims-test
    Then the body contains "contoso"
    And the body does not contain "acme"

  Scenario: Entry claims are updated via PUT
    When I PUT /v1/entries/skill/test-skill/claims with {"claims":{"team":"eng"}}
    Then the response status is 204

  # ── Managed source limit ──────────────────────────────────────────────────

  Scenario: Second managed source is rejected with 409
    Given the managed-test source still exists
    When I PUT /v1/sources/second-managed with body {"managed":{}}
    Then the response status is 409
    And the body contains "at most one managed source is allowed"

  # ── List completeness ─────────────────────────────────────────────────────

  Scenario: All created sources appear in list
    Given three file-data sources are created: smoke-src-a, smoke-src-b, smoke-src-c
    When I GET /v1/sources
    Then the response status is 200
    And the body contains "smoke-src-a"
    And the body contains "smoke-src-b"
    And the body contains "smoke-src-c"

  Scenario: All created registries appear in list
    Given three registries are created referencing the file-data sources
    When I GET /v1/registries
    Then the response status is 200
    And the body contains "smoke-reg-a"
    And the body contains "smoke-reg-b"
    And the body contains "smoke-reg-c"

  # ── OAuth / auth enforcement ───────────────────────────────────────────────
  # The stack is restarted with auth.mode: oauth before these scenarios run.
  # No real OIDC provider is needed: "missing token" and "malformed token"
  # cases are rejected before JWKS is consulted.

  Scenario: Public paths are accessible without a token in OAuth mode
    Given the server is restarted in OAuth mode
    When I GET /health
    Then the response status is 200
    When I GET /readiness
    Then the response status is 200
    When I GET /version
    Then the response status is 200
    When I GET /openapi.json
    Then the response status is 200

  Scenario: OAuth protected-resource metadata is publicly accessible
    When I GET /.well-known/oauth-protected-resource
    Then the response status is 200
    And the body contains "authorization_servers"

  Scenario: MCP list-servers requires a token in OAuth mode
    When I GET /registry/default/v0.1/servers without an Authorization header
    Then the response status is 401
    And the response includes a WWW-Authenticate header with Bearer scheme
    And the WWW-Authenticate header contains resource_metadata

  Scenario: Admin registries endpoint requires a token
    When I GET /v1/registries without an Authorization header
    Then the response status is 401

  Scenario: Admin sources endpoint requires a token
    When I GET /v1/sources without an Authorization header
    Then the response status is 401

  Scenario: Malformed Bearer token returns 401 invalid_token
    When I GET /registry/default/v0.1/servers with Authorization: Bearer not-a-real-jwt
    Then the response status is 401
    And the WWW-Authenticate header contains error="invalid_token"

Read the full file on GitHub · 753 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 5d ago First seen · 753 lines · 40 tokens per session scan B bd60fc8cb84c

Subscribe to this mod's changes

smoke-test is a skill published in the GitHub repository stacklok/toolhive-registry-server (24 stars, last pushed yesterday), licensed Apache-2.0. It adds 40 tokens to every session and 8,653 once invoked, about $0.0002 per session on Opus 5. A static security scan graded it B with 2 findings (sends data to an external url, makes network calls). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.