AI/ML Model Testing

AI/ML Model Testing is a skill for Claude Code, Codex from PramodDutta/qaskills. It costs 29 tokens per session (790 once invoked), scanned A, original, MIT.

A testing guide for machine-learning models, including checks for accuracy, unfair differences between groups, changing real-world data, experiments between model versions, and regressions.

In plain words
What is it for?
Use it to set up model tests, investigate AI-related failures, compare model versions, monitor data drift, check for bias, and connect verification to CI/CD pipelines.
Why use it?
It helps reveal when a model becomes less accurate, behaves unfairly, stops matching current data, or breaks after a change. It also encourages clear results that developers can act on and repeat in automated build checks.

Skill for Claude CodeCodex

Which agent this was written for is unclear — built for aider. Also seen: mentions Codex; built for aider.

Good fit Use it to set up model tests, investigate AI-related failures, compare model versions, monitor data drift, check for bias, and connect verification to CI/CD pipelines.

Compare 6 skills from other repositories ↓
Install with agentmods
npx agentmods add skills/pramoddutta/qaskills/ai-model-testing
Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

Any agent
npx skills add PramodDutta/qaskills --skill ai-model-testing
Clone the repo
git clone --depth 1 https://github.com/PramodDutta/qaskills

Made for: Claude Code, Codex.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for AI/ML Model Testing

README.md
[![agentmods](https://agentmods.dev/badge/skills/pramoddutta/qaskills/ai-model-testing/github.svg)](https://agentmods.dev/skills/pramoddutta/qaskills/ai-model-testing)
Your own site
<a href="https://agentmods.dev/skills/pramoddutta/qaskills/ai-model-testing"><img src="https://agentmods.dev/badge/skills/pramoddutta/qaskills/ai-model-testing/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

agentmods 80×15 button for AI/ML Model Testing

Your own site · 80×15
<a href="https://agentmods.dev/skills/pramoddutta/qaskills/ai-model-testing"><img src="https://agentmods.dev/badge/skills/pramoddutta/qaskills/ai-model-testing.svg" alt="Reviewed on agentmods" width="80" height="20"></a>
Per session 29 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 790 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe. Third-party audits
  • NVIDIA SkillSpector pass 7 Sept 2026
How audits are shown
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00029 $0.00790
Opus 5.5 $0.00012 $0.00316
Sonnet 5.5 $0.00006 $0.00158
Haiku 4.5 $0.00003 $0.00079

Measured 1mo ago against content hash f919f72bc25b, method: parsed. Prices are Anthropic first-party input rates as of 2026-10-07, from the pricing page.

Security

Grade A, and why

AI/ML Model Testing scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 1mo ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

seed-skills/ai-model-testing/SKILL.md · 89 lines

How it starts

The opening of the file, as written. The whole thing — 89 lines — stays where its author put it; the contents beside it link to each section on GitHub.

AI/ML Model Testing

You are an expert QA engineer specializing in ai/ml model testing. When the user asks you to write, review, debug, or set up ai related tests or configurations, follow these detailed instructions.

Core Principles

  1. Quality First — Ensure all ai implementations follow industry best practices and produce reliable, maintainable results.
  2. Defense in Depth — Apply multiple layers of verification to catch issues at different stages of the development lifecycle.
  3. Actionable Results — Every test or check should produce clear, actionable output that developers can act on immediately.
  4. Automation — Prefer automated approaches that integrate seamlessly into CI/CD pipelines for continuous verification.
  5. Documentation — Ensure all ai configurations and test patterns are well-documented for team understanding.

When to Use This Skill

  • When setting up ai for a new or existing project
  • When reviewing or improving existing ai implementations
  • When debugging failures related to ai
  • When integrating ai into CI/CD pipelines
  • When training team members on ai best practices

Implementation Guide

Setup & Configuration

When setting up ai, follow these steps:

  1. Assess the project — Understand the tech stack (python) and existing test infrastructure
  2. Choose the right tools — Select appropriate ai tools based on project requirements
  3. Configure the environment — Set up necessary configuration files and dependencies
  4. Write initial tests — Start with critical paths and expand coverage gradually
  5. Integrate with CI/CD — Ensure tests run automatically on every code change

Best Practices

  • Keep tests focused — Each test should verify one specific behavior or requirement
  • Use descriptive names — Test names should clearly describe what is being verified
  • Maintain test independence — Tests should not depend on execution order or shared state
  • Handle async operations — Properly await async operations and use appropriate timeouts
  • Clean up resources — Ensure test resources are properly cleaned up after execution

Read the full file on GitHub · 89 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 1mo ago First seen · 89 lines · 29 tokens per session scan A f919f72bc25b

Subscribe to this mod's changes

AI/ML Model Testing is a skill published in the GitHub repository PramodDutta/qaskills (238 stars, last pushed today), licensed MIT. It adds 29 tokens to every session and 790 once invoked, about $0.0001 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Related

Other skills, from other repositories

data-quality-frameworks

Implement data quality validation with Great Expectations, dbt tests, and data contracts. Use when building data quality pipelines, implementing validation rules, or establishing data contracts.

wshobson/agents · 37 tokens

deep-learning-pytorch

Use when writing or changing PyTorch code — model, Dataset/DataLoader, training and evaluation loops, determinism, mixed precision, torch.compile, checkpoints with safetensors, Hugging Face models, ONNX export, and tests that run on CPU.

makifbaysal/tasktrooper · 55 tokens

leakage-safe-feature-engineering

Use when building or changing features, preprocessing, encoders, resampling or train/validation/test splits — keep every fitted transform inside a Pipeline fitted on training folds only, and build time and group features without seeing the future or the same entity twice.

makifbaysal/tasktrooper · 57 tokens

autoresearch

Autonomously optimize any Claude Code skill by running it repeatedly, scoring outputs against binary evals, mutating the prompt, and keeping improvements. Based on Karpathy's autoresearch methodology. Use when: optimize this skill, improve this skill, run autoresearch on, make this skill better, self-improve skill…

adriannoes/awesome-agentic-ai · 100 tokens

hunt-race-condition

Hunting skill for race condition vulnerabilities. Built from 3 public bug bounty reports. Use when hunting race condition on any target.

adriannoes/awesome-agentic-ai · 31 tokens

assessing-vector-and-embedding-weaknesses

Test vector stores for embedding inversion, cross-tenant leakage, and poisoning.

adriannoes/awesome-agentic-ai · 26 tokens