conformance

A guide for running and maintaining a pinned conformance test suite. Conformance tests check whether a system follows a defined protocol or standard; here, the suite checks the fold gateway against the official MCP tests.

In plain words
What is it for?
Use it to run the 40-check suite, compare gateway behavior with the upstream service, investigate failures such as streaming or error handling, and update the pinned versions when needed.
Why use it?
It helps distinguish a gateway defect from a changed upstream test or fixture when a check fails. It also keeps the locally tested versions deliberately fixed until they are intentionally updated.

Skill for Claude CodeCodex

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add skills/fold-run/fold/conformance
Any agent
npx skills add fold-run/fold --skill conformance
Clone the repo
git clone --depth 1 https://github.com/fold-run/fold

Made for: Claude Code, Codex.

Per session 52 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 581 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00052 $0.00581
Opus 5 $0.00026 $0.00291
Sonnet 5 $0.00010 $0.00116
Haiku 4.5 $0.00005 $0.00058

Measured yesterday against content hash eb2aec452c26, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

conformance scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured yesterday.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

.claude/skills/conformance/SKILL.md · 47 lines

How it starts

The opening of the file, as written. The whole thing — 47 lines — stays where its author put it; the contents beside it link to each section on GitHub.

fold conformance suite

make conformance runs the official MCP conformance suite through the gateway fronting the conformance repo's everything-server in passthrough mode. CI gates merges on 40/40 checks. Requires go, node/npm/npx, git.

The pin lives in scripts/conformance.sh:

  • CONFORMANCE_COMMIT — conformance repo commit (server fixture)
  • CONFORMANCE_PKG — npm package version (the checker)

The scheduled drift workflow overrides these to main/@latest to catch upstream movement early; the pin itself is bumped deliberately.

Debugging a failure

  1. Reproduce locally: make conformance. Note which check fails.
  2. First question: did the gateway stop being invisible? Run the same scenario against the upstream directly (the script starts it on UPSTREAM_PORT, default 3901; gateway on 3902) and diff the behavior.
    • Differs through the gateway only → a fold bug. Usual suspects: response buffering/rewriting that federation doesn't require, namespacing applied in passthrough mode (it must not be), SSE stream handling, or a minted error replacing a pass-through error.
    • Same failure hitting the upstream directly → the suite or fixture moved; this is a pin/drift issue, not a fold bug.
  3. Environment failures (ports busy, npx cache, node missing) look like hangs — check wait_for timeouts in the script output before suspecting the gateway.

Bumping the pin (deliberate, never drive-by)

  1. Test the target first without editing anything: CONFORMANCE_COMMIT=<sha> CONFORMANCE_PKG=@modelcontextprotocol/conformance@<ver> make conformance
  2. If the check count changed (e.g. new checks beyond 40), read the new checks and decide: does fold pass them, or does a gap need closing first? A gap that stays open belongs in README "Not implemented".
  3. Edit both pin variables in scripts/conformance.sh together, update the 40/40 count in CLAUDE.md/CI docs if it changed, and run the full suite once more on the edited script.
  4. The bump gets its own commit with the reason (drift workflow finding, new spec feature, etc.) — per-step approval before committing.

Read the full file on GitHub · 47 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. yesterday First seen · 47 lines · 52 tokens per session scan A eb2aec452c26

Subscribe to this mod's changes

conformance is a skill published in the GitHub repository fold-run/fold (1 stars, last pushed 2d ago), licensed Apache-2.0. It adds 52 tokens to every session and 581 once invoked, about $0.0003 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.

Related

Other skills, from other repositories

hotplex-release

通过判断 SemVer 影响、整理 CHANGELOG、对账版本面和验证发布输入来准备或发布 HotPlex 版本。不要用于主机二进制更新、运行时诊断或文档巡逻。.

hrygo/hotplex · 52 tokens

hotplex-docs-patrol

维护 HotPlex 当前文档,将代码、配置、API 或发布变更映射到 BFS 可达文档,只修复已验证的文档漂移。不要用于普通润色、运行时诊断或版本发布。.

hrygo/hotplex · 57 tokens

hotplex-stt-tts

初始化或修复 HotPlex 本地 STT 和 MOSS TTS 运行时,包括 Python 依赖、官方模型、配置和验收。仅在明确授权主机变更时使用;只读检查请使用 hotplex-cli 或 hotplex-diagnostics。.

hrygo/hotplex · 65 tokens

hotplex-diagnostics

深入诊断 HotPlex Gateway、Worker、Session、日志或反馈链异常。普通 status/doctor/security/config 只读检查属于 hotplex-cli;安装、更新、重启、配置写入和 Admin 变更属于 hotplex-operator。.

hrygo/hotplex · 59 tokens

hotplex-cli

使用 HotPlex CLI 处理 Cron、明确请求的 Slack 操作、普通用户聊天命令指引,以及只读 status、doctor、security、config 诊断。不要用于飞书写操作、发布、服务安装、二进制更新或 Admin 变更。.

hrygo/hotplex · 63 tokens

hotplex-operator

运维或初始化 HotPlex 主机,覆盖首次 onboard、服务安装/启动、二进制更新、主机配置、审计检查和 Admin 变更。仅在明确授权的 operator 上下文中使用。.

hrygo/hotplex · 54 tokens