Plugin Claude Code
Plugin marketplace listing 1 plugin: agent-reliability.
2 2mo ago A
tokens not measured
original MIT
Plugin Claude Code
Claude skills for AI agent and ML reliability: reproduce the eval-to-production gap, catch confidently-wrong outputs, and prove root cause with verified numbers. Start with production-autopsy.
2 2mo ago A
tokens not measured
original MIT
Skill Claude CodeCodex
Detects and quantifies confidently-wrong behavior: where a model, classifier, agent, or LLM judge is highly confident and incorrect, and where the coupling between confidence and correctness breaks down under distribution shift. Measures calibration on the in-distribution set and again on the production distribution…
2 2mo ago A 208 tokens
original MIT
Skill Claude CodeCodex
Start here. Audits a deployed ML or LLM or agent system that scores well on evaluation but fails, regresses, or behaves unexpectedly in production. Runs a reproducible root-cause "autopsy": frames the eval-to-deployment gap, reproduces the production failure, quantifies it by slice, tests confidence calibration under…
2 2mo ago A 258 tokens
original MIT
tool-eval
05Skill Claude CodeCodex
Verifies the tools an agent or orchestrator depends on, by re-deriving a tool evaluation's real accuracy instead of trusting a single pass/fail score. Separates a formatting miss (a correct value scored wrong) from a real failure (a wrong value scored right), recomputes the expected answer from the raw inputs rather…
2 2mo ago A 253 tokens
original MIT
Skill Claude CodeCodex
Evaluates whether an agent or a multi-step or multi-agent system actually completes the task, not just whether each step or the final answer looks right. Captures full trajectories (every step, tool call, and intermediate state), measures per-step success against end-to-end success, exposes the compounding-error curve…
2 2mo ago A 214 tokens
original MIT