hamelsmu

hamelsmu/evals-skills

Plugin Claude Code

Plugin marketplace listing 1 plugin: evals-skills.

hamelsmu/evals-skills

Plugin Claude Code

Skills for building LLM evaluations: pipeline audit, error analysis, synthetic data generation, LLM-as-Judge design, evaluator validation, RAG evaluation, and annotation interfaces.

hamelsmu/evals-skills

Skill Claude CodeCodex

Build a custom browser-based annotation interface tailored to your data for reviewing LLM traces and collecting structured feedback. Use when you need to build an annotation tool, review traces, or collect human labels.

hamelsmu/evals-skills

Skill Claude CodeCodex

Help the user systematically identify and categorize failure modes in an LLM pipeline by reading traces. Use when starting a new eval project, after significant pipeline changes (new features, model switches, prompt rewrites), when production metrics drop, or after incidents.

hamelsmu/evals-skills

Skill Claude CodeCodex

Audit an LLM eval pipeline and surface problems: missing error analysis, unvalidated judges, vanity metrics, etc. Use when inheriting an eval system, when unsure whether evals are trustworthy, or as a starting point when no eval infrastructure exists. Do NOT use when the goal is to build a new evaluator from scratch…

hamelsmu/evals-skills

Skill Claude CodeCodex

Guides evaluation of RAG pipeline retrieval and generation quality. Use when evaluating a retrieval-augmented generation system, measuring retrieval quality, assessing generation faithfulness or relevance, generating synthetic QA pairs for retrieval testing, or optimizing chunking strategies.

hamelsmu/evals-skills

Skill Claude CodeCodex

Create diverse synthetic test inputs for LLM pipeline evaluation using dimension-based tuple generation. Use when bootstrapping an eval dataset, when real user data is sparse, or when stress-testing specific failure hypotheses. Do NOT use when you already have 100+ representative real traces (use stratified sampling…

hamelsmu/evals-skills

Skill Claude CodeCodex

Calibrate an LLM judge against human labels using data splits, TPR/TNR, and bias correction. Use after writing a judge prompt (write-judge-prompt) when you need to verify alignment before trusting its outputs. Do NOT use for code-based evaluators (those are deterministic; test with standard unit tests).

hamelsmu/evals-skills

Skill Claude CodeCodex

Design LLM-as-Judge evaluators for subjective criteria that code-based checks cannot handle. Use when a failure mode requires interpretation (tone, faithfulness, relevance, completeness). Do NOT use when the failure mode can be checked with code (regex, schema validation, execution tests). Do NOT use when you need to…

hamelsmu/claude-review-loop

Plugin Claude Code

Plugin marketplace listing 1 plugin: review-loop.

hamelsmu/claude-review-loop

Plugin Claude Code

Automated code review loop: Claude implements, Codex reviews independently, Claude addresses feedback.

hamelsmu/claude-review-loop

Command

Cancel an active review loop.

hamelsmu/claude-review-loop

Command

Start a review loop: implement task, get independent Codex review, address feedback.

hamelsmu/claude-review-loop

Hook

Runs when the agent finishes a response, executing stop-hook.sh. From hamelsmu/claude-review-loop.

hamelsmu/design-research

Skill Claude CodeCodex

Conduct comprehensive design research on any website or web application using a team of browser-automation agents. Use when asked to analyze, audit, or document a website's design system, UX patterns, layout, typography, colors, components, accessibility, responsive behavior, and technical implementation.

hamelsmu/website-to-api

Plugin Claude Code

Reverse-engineer website internal APIs using Chrome browser automation. Discover endpoints, extract auth, and build CLI scripts for any website.

hamelsmu/website-to-api

Instructions file CodexOpenCode

AGENTS.md instructions for hamelsmu/website-to-api, covering website-to-api, structure, adding a new site skill and security.

hamelsmu/website-to-api

Skill Claude CodeCodex

Retrieve posts from any Substack newsletter using its internal API. Use when asked to list, fetch, search, or download Substack articles/posts. Supports pagination, search, sorting, and full content retrieval for paid posts.

hamelsmu/website-to-api

Skill Claude CodeCodex

Reverse-engineer a website's internal API using Chrome browser automation. Use when asked to discover, wrap, or build a programmatic interface for a website that doesn't have a public API. Guides the process of finding endpoints, extracting auth, and building scripts.

hamelsmu/reverse-eng-site-skill

Skill Claude CodeCodex

Reverse-engineer a website's undocumented internal API by instrumenting browser traffic, replaying captured requests as direct HTTP, proving auth and response fidelity, and packaging a resilient client or site-specific skill. Use when asked to discover, automate, wrap, download from, or repair an integration with a…

hamelsmu/reverse-eng-site-skill

Skill Claude CodeCodex

Operate SITE through its verified internal API. Use when asked to OPERATIONS.