OpenMed is local-first healthcare AI software that extracts clinical information and removes personally identifying details from clinical text on hardware controlled by the user. Healthcare developers use its Python runtime, Apple Silicon and mobile SDKs, and browser support for on-device clinical NER and PII de-identification.
Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add maziyarpanahi/openmed --skill evaluating-with-leakage-gatesgit clone --depth 1 https://github.com/maziyarpanahi/openmedWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/maziyarpanahi/openmed/evaluating-with-leakage-gates)<a href="https://agentmods.dev/skills/maziyarpanahi/openmed/evaluating-with-leakage-gates"><img src="https://agentmods.dev/badge/skills/maziyarpanahi/openmed/evaluating-with-leakage-gates/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/maziyarpanahi/openmed/evaluating-with-leakage-gates"><img src="https://agentmods.dev/badge/skills/maziyarpanahi/openmed/evaluating-with-leakage-gates.svg" alt="Reviewed on agentmods" width="80" height="20"></a>- NVIDIA SkillSpector pass
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00158 | $0.02189 |
| Opus 5 | $0.00079 | $0.01094 |
| Sonnet 5 | $0.00032 | $0.00438 |
| Haiku 4.5 | $0.00016 | $0.00219 |
Grade A, and why
evaluating-with-leakage-gates scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 12d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 165 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Evaluating with Leakage Gates
OpenMed's release gates answer one question: did any PHI leak? A de-id model
with a beautiful F1 can still leak a single SSN — and that one leak is a HIPAA
breach. So openmed.eval gates on residual leakage and per-label recall
floors, not on aggregate F1. The candidate is either RELEASABLE or
QUARANTINED; there is no partial credit.
When to use this skill
- You have a candidate de-id or PII model and need a ship / no-ship decision.
- You want to run the benchmark harness over a synthetic golden suite.
- You need to enforce direct-identifier recall floors and
critical_leakage == 0. - You need calibration thresholds (
thresholds.json) before the gate will pass. - You want a signed, reproducible gate report for governance.
This is the flagship eval skill. For a pure NER scorecard see
benchmarking-clinical-ner; for CI wiring see gating-deid-leakage.
The gates (G1a–G8)
| Gate | Checks | Floor / rule |
|---|---|---|
| G1a | Direct & quasi identifiers (PERSON, EMAIL, PHONE, SSN, ID_NUM, DATE_OF_BIRTH, ...) | recall ≥ 0.990 (v1.6) / 0.995 (v2.0); strict-no-leak policies raise the floor |
| G1b | Structured secrets (API_KEY, ACCOUNT_NUMBER, CREDIT_CARD, IBAN) | recall ≥ 0.995 |
| G2 | Free-text names/locations/dates | recall ≥ 0.980 (v1.6) / 0.990 (v2.0) |
| G3 | Critical leakage (SSN, CREDIT_CARD, CVV, API_KEY, PIN, IBAN, ...) | count must be exactly 0 |
| G4 | Quantized recall delta vs fp parent | within INT8 / INT4 limits |
| G5 | Latency & RAM vs device tier budget | p50/p95/RAM under tier budget |
| G6 | p50/p95 latency documented | must be present and finite |
| G7 | Baseline regression | recall drop ≤ 0.002/label; leakage ≤ soft ceiling 0.005 and ≤ steward target; no leakage regression vs last-green |
| G8 | Span integrity | predicted spans validate (no overlaps/out-of-range) |
Constants live in openmed.eval.release_gates (G1A_V16_RECALL_FLOOR,
G1B_RECALL_FLOOR, G7_RECALL_DROP_LIMIT, RESIDUAL_LEAKAGE_SOFT_CEILING, ...).
Confirm them there rather than hardcoding — they move per milestone.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 12d ago First seen · 165 lines · 158 tokens per session scan A d4f373b17535
evaluating-with-leakage-gates is a skill published in the GitHub repository maziyarpanahi/openmed (5,290 stars, last pushed yesterday), licensed Apache-2.0. It adds 158 tokens to every session and 2,189 once invoked, about $0.0008 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
field-test
Exercise tools, resources, and prompts against a live HTTP server via MCP JSON-RPC over curl. Starts the server, surfaces the catalog, runs real and adversarial inputs, and produces a tight report with concrete findings and numbered follow-up options. Use after adding or modifying definitions, or when the user asks to…
api-testing
Testing patterns for MCP tool/resource handlers using createMockContext and Vitest. Covers mock context options, handler testing, McpError assertions, format testing, Vitest config setup, and test isolation conventions.
add-test
Scaffold a test file for an existing tool, resource, or service. Use when the user asks to add tests, improve coverage, or when a definition exists without a matching test file.
rust-testing
Rust testing patterns including unit tests, integration tests, async testing, property-based testing, mocking, and coverage. Follows TDD methodology.
golang-testing
Go testing best practices including table-driven tests, test helpers, benchmarking, race detection, coverage analysis, and integration testing patterns. Use when writing or improving Go tests.
verification-loop
A comprehensive verification system for Claude Code sessions. Use when verifying a Claude Code session's work before claiming it is complete.