opendatahub-io

opendatahub-io/agent-eval-harness

Plugin Claude Code

Plugin marketplace listing 1 plugin: agent-eval-harness.

opendatahub-io/agent-eval-harness

Plugin Claude Code

Agent and skill evaluation harness with MLflow integration.

opendatahub-io/agent-eval-harness

Instructions file CodexOpenCode

Instructions for opendatahub-io/agent-eval-harness.

opendatahub-io/agent-eval-harness

Instructions file

Instructions for opendatahub-io/agent-eval-harness, covering agent eval harness, project status, execution model, execution mode (case vs batch) and what to execute (skill vs prompt).

opendatahub-io/agent-eval-harness

Skill Claude CodeCodex

Generate eval.yaml for the agent eval harness. Two modes - (1) Skill-based - examines SKILL.md, sub-skills, scripts, test cases to verify implementation quality, OR (2) Prompt-based - tests agent capabilities using custom analysis prompts (documentation effectiveness, pattern understanding, API usage, constraint…

opendatahub-io/agent-eval-harness

Skill Claude CodeCodex

Run Design-of-Experiments (DoE) evaluations with ANOVA over a matrix of agent configurations — comparing models, thinking-effort levels, prompts, or other factors across shared test cases, with repeated-measures / mixed-effects statistics that account for case difficulty plus a cost/quality Pareto view. Use whenever…

opendatahub-io/agent-eval-harness

Skill Claude CodeCodex

Evaluate the full harness configuration as a system. Scans all skills, commands, CLAUDE.md, and hooks for redundancy, overlap, type misclassification, and structural issues. Produces an informational report with restructuring suggestions. Use when the user wants to check their overall setup health, find redundant…

opendatahub-io/agent-eval-harness

Skill Claude CodeCodex

Compare evaluation results across multiple models or runs. Takes a directory of eval run artifacts (summary.yaml, runresult.json, HTML reports) and produces a tabbed HTML comparison report with model cards, quality/cost tables, per-case breakdowns, and embedded original reports. Use when the user wants to compare…

opendatahub-io/agent-eval-harness

Skill Claude CodeCodex

Generate evaluation test cases for an eval.yaml. Sources cases per generation.strategy - skill analysis (default), synthetic LLM generation from generation prompts (documentation and agent-capability evals), or MLflow production traces. Bootstraps a starter dataset or augments an existing one to improve coverage. Use…

opendatahub-io/agent-eval-harness

Skill Claude CodeCodex

MLflow integration for evaluation — sync datasets, log run results, push/pull feedback between the harness and MLflow traces. Use when the user wants to log eval results to MLflow, sync test cases to MLflow datasets, connect judge scores to traces, pull MLflow annotations for eval-optimize, or view results in the…

opendatahub-io/agent-eval-harness

Skill Claude CodeCodex

Automated skill improvement loop. Runs eval, identifies judge failures, reads traces and rationale, edits the SKILL.md to fix issues, re-runs to verify, and checks for regressions. Use when the user wants to automatically improve a skill based on eval results, fix failing judges, make the skill better, auto-fix…

opendatahub-io/agent-eval-harness

Skill Claude CodeCodex

Interactive review of evaluation results. Presents judge scores and skill outputs for human feedback, then proposes SKILL.md improvements based on what the user identifies. Use when the user wants to review eval results, look at results, check scores, see what went wrong, give qualitative feedback on skill outputs, or…

opendatahub-io/agent-eval-harness

Skill Claude CodeCodex

Execute an evaluation against test cases (skill or prompt mode), score with judges, and report results. Requires eval.yaml (generated by /eval-analyze). Use when the user wants to test a skill, run eval, benchmark, compare models, detect regressions, check skill quality, or verify changes didn't break anything.…

opendatahub-io/agent-eval-harness

Skill Claude CodeCodex

Optional environment configurator for the agent-eval-harness. Configures MLflow tracking, verifies API keys, and troubleshoots dependency issues. Detects available skills and agentic documentation (CLAUDE.md, AGENTS.md, ai-docs/) to suggest appropriate evaluation modes. Not required for basic usage — dependencies…

opendatahub-io/agent-eval-harness

Skill Claude CodeCodex

Query Jira for bugs in a project and summarize coverage gaps. Test fixture for e2e external-state field detection.

opendatahub-io/ai-helpers

Plugin Claude Code

Plugin marketplace listing 19 plugins: fips-compliance-checker, autofix-skills, odh-ai-helpers, odh-code-quality, odh-documentation.

opendatahub-io/ai-helpers

Instructions file CodexOpenCode

Instructions for opendatahub-io/ai-helpers, covering ai helpers marketplace, repository purpose, tool types, skills and agents.

opendatahub-io/ai-helpers

Instructions file

Instructions for opendatahub-io/ai-helpers, a project described as: AI Helpers — collections of Skills, Hooks, Agents (compatible with Claude Code, Cursor and other tools) and Gemini Gems.

opendatahub-io/ai-helpers

Plugin Claude Code

opendatahub-io/ai-helpers

Agent

../../odh-python-packaging/agents/python-packaging-investigator.md.

opendatahub-io/ai-helpers

Hook

Runs when a session starts on startup, executing deprecation-notice.sh. From opendatahub-io/ai-helpers.

opendatahub-io/ai-helpers

Plugin Claude Code

CodeRabbit review triage and project-conformant unit test generation.

opendatahub-io/ai-helpers

Skill Claude CodeCodex

Use when you need to evaluate CodeRabbit PR comments and fix or reply.

opendatahub-io/ai-helpers

Skill Claude CodeCodex

Use this skill to write unit tests that strictly conform to the project's existing testing structure, patterns, and style by learning from similar tests before writing anything new.