inspect_evals: Instructions file for Claude Code

Claude Code Instructions

This is the inspect_evals repository - a collection of evaluation tasks for the Inspect AI framework.

Running Commands

This project uses uv (by Astral) for package management. Do not use pip install, python -m venv, source .venv/bin/activate, or bare python/pytest commands. Always use uv to run commands:

• uv sync — install/sync dependencies (replaces pip install -e .)
• uv run pytest ... — run tests (not pytest or python -m pytest)
• uv run python ... — run Python scripts (not python or python3)
• uv run inspect eval ... — run evaluations
• uv run ruff ... — run the linter
• uv run mypy ... — run the type checker

Note: uv is Astral's Python package manager. It is not related to uvicorn (an ASGI web server) — do not confuse them.

Contributing

For development setup, submission requirements, and contribution guidelines, see CONTRIBUTING.md.

Coding Style

When writing or modifying code in this repository, follow the guidelines in BEST_PRACTICES.md. Pay particular attention to the Writing comments section before adding any comments.

Markdown prose must not be hard-wrapped - write each paragraph as a single line. This is enforced by the mdformat pre-commit hook (configured in .mdformat.toml); see the markdown rules in AGENTS.md. Running uv run make check before completing your code ensures all markdown files are correctly formatted.

Evaluation Checklist

When creating or reviewing evaluations, refer to EVALUATION_CHECKLIST.md.

Versioning

For when to bump the inspect evals changelog, see PACKAGE_VERSIONING.md

For when to bump an inspect evals task version, see TASK_VERSIONING.md

Workflows

For common workflows (fixing evals, reviewing evals, making evaluation reports, checking agent trajectories, etc.), see AGENTS.md.