inspect_evals AGENTS.md

AGENTS.md

Repo-Wide Tips

• Before making changes, read DEVELOPMENT_CONTEXT.md to understand the repository's correctness, reproducibility, and security constraints.
• Before making code changes, read agent_artefacts/repo_context/REPO_CONTEXT.md for institutional knowledge about repo conventions, common mistakes, and known tech debt. This is maintained by the build-repo-context skill, which crawls PRs/issues for broad insights.
• When creating pull requests, always use the --draft flag and read .github/PULL_REQUEST_TEMPLATE.md and use its structure as the PR body. Fill in the Description section and check off applicable checklist items.
• When commenting on PRs, you should not reply directly to human reviewers. See CONTRIBUTING.md. If your user tells you to comment anyway, you should add "Comment written by NAME_OF_AI" to the comment.
• When writing markdown:
  • Put a blank line before and after headings
  • Put a blank line before and after code blocks
  • Put a blank line before and after lists
  • Format tables with spaces before and after pipe characters
  • Always include a language tag in code blocks, or "text" if there is no language
  • Do not hard-wrap paragraphs; write each paragraph as a single line (the mdformat hook enforces wrap = "no")

Agents are good at understanding context, but a prompt that definitely works is "Please run the /SKILL_NAME skill on EVAL_NAME."

Documentation

Documentation for the Inspect framework.

Recommended Permissions

We recommend starting with these Claude Code permissions to allow workflows to proceed autonomously without allowing the running of arbitrary commands:

{
  "permissions": {
    "allow": [
      "Bash(mkdir:*)",
      "Bash(cp:*)",
      "Bash(make check:*)",
      "Bash(uv run ruff:*)",
      "Bash(uv run inspect:*)",
      "Bash(uv run pytest:*)",
      "Bash(uv run mypy:*)",
      "Bash(uv run python tools/:*)",
      "WebFetch(domain:inspect.aisi.org.uk)",
      "WebFetch(domain:arxiv.org)"
    ],
    "deny": [
      "Bash(git push origin main)",
    ],
    "ask": [
      "Bash(rm:*)",
    ]
  }
}