What is datarobot-agent-tester AGENTS.md?

Project instructions for datarobot-agent-tester, a Python library and command-line tool for creating, testing, and improving AGENTS.md files and AI coding skills. It describes the project layout, important files, and development commands.

Use it when modifying agent-instruction generation, skill evaluation, configuration, the command-line interface, or the test suite.

It gives coding agents the project-specific rules they need before changing the library, command-line tool, or tests.

How datarobot-oss/datarobot-agent-tester configures Codex and OpenCode

Reuse

Borrowing it

Nothing to install: this file belongs to datarobot-oss/datarobot-agent-tester. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.

Copy the file

curl -O https://raw.githubusercontent.com/datarobot-oss/datarobot-agent-tester/main/AGENTS.md

Clone the repo

git clone --depth 1 https://github.com/datarobot-oss/datarobot-agent-tester

Made for: Codex, OpenCode.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

README.md

[![agentmods](https://agentmods.dev/badge/instructions/datarobot-oss/datarobot-agent-tester/agents-md/github.svg)](https://agentmods.dev/instructions/datarobot-oss/datarobot-agent-tester/agents-md)

Your own site

<a href="https://agentmods.dev/instructions/datarobot-oss/datarobot-agent-tester/agents-md"><img src="https://agentmods.dev/badge/instructions/datarobot-oss/datarobot-agent-tester/agents-md/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

Your own site · 80×15

<a href="https://agentmods.dev/instructions/datarobot-oss/datarobot-agent-tester/agents-md"><img src="https://agentmods.dev/badge/instructions/datarobot-oss/datarobot-agent-tester/agents-md.svg" alt="Reviewed on agentmods" width="80" height="20"></a>

Per session 691 This file is loaded in full into every session.

When invoked 691 The same file — it is already loaded in full.

Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe.

Origin original No closer match found in the catalogue.

Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

Model	Per session	Once invoked
Fable 5.1	$0.00691	$0.00691
Opus 5	$0.00345	$0.00345
Sonnet 5	$0.00138	$0.00138
Haiku 4.5	$0.00069	$0.00069

Measured 9d ago against content hash 9ccad7d226ed, method: parsed. Prices are Anthropic first-party input rates as of 2026-09-09, from the pricing page.

Security

Grade A, and why

datarobot-agent-tester AGENTS.md scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 9d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

AGENTS.md · 67 lines

How it starts

The opening of the file, as written. The whole thing — 67 lines — stays where its author put it; the contents beside it link to each section on GitHub.

datarobot-agent-tester

Python library and CLI for generating, testing, and improving AGENTS.md files and AI coding skills via the DataRobot LLM gateway.

Layout

dr_agents_tester/   Python package (flat layout, no src/)
  __init__.py       Public API: Config, AgentsMd, Skills
  config.py         Config dataclass + env var defaults
  llm.py            litellm wrapper for DataRobot gateway
  context.py        Git file tree + priority file gathering
  agents_md.py      AgentsMd class: generate / test / revise / copilot
  skills.py         Skills class: test / improve / test_all
  cli.py            CLI entry point (dr-agent command)
skills/             Example skill files (.md) — test targets
tests/              pytest suite (no LLM calls — all mocked)
  fixtures/         Sample AGENTS.md and skill for tests
.github/workflows/
  ci.yml            Lint + test on every push/PR (3.11, 3.12, 3.13)
  agents-md.yml     Generate/test AGENTS.md and skills (needs DATAROBOT_API_TOKEN)

Key files to modify

dr_agents_tester/agents_md.py — AGENTS.md prompts and logic
dr_agents_tester/skills.py — skill evaluation/improvement prompts and logic
dr_agents_tester/cli.py — CLI subcommands
skills/*.md — the actual skill files (modify or add new ones here)

Do NOT modify files in tests/fixtures/ without updating the relevant tests.

Commands

task install       # uv sync --all-groups
task lint          # ruff format + check + mypy (modifies files)
task lint-check    # lint check only, no modifications (used in CI)
task test          # pytest with coverage

task agents-generate   # dr-agent agents generate
task agents-test       # dr-agent agents test
task agents-revise     # dr-agent agents revise
task skills-test       # dr-agent skills test --all
task skills-improve    # dr-agent skills improve --skill <path>

Pass extra flags via -- <flags>, e.g. task agents-generate -- --dry-run.

Conventions

All LLM calls go through llm.call_llm(prompt, model, config) — never call litellm directly
Tests mock dr_agents_tester.agents_md.call_llm and dr_agents_tester.skills.call_llm
Marker constants MARKER_START / MARKER_END live in agents_md.py — import from there
Config reads env vars by default; always call cfg.validate() before use in CLI commands
Skills report files are named <stem>.skill-report.md (REPORT_SUFFIX constant in skills.py)
AGENTS.md report files are named .agents-md-report.md

Read the full file on GitHub · 67 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

9d ago First seen · 67 lines · 691 tokens per session scan A 9ccad7d226ed

Subscribe to this mod's changes

datarobot-agent-tester AGENTS.md is an instructions file published in the GitHub repository datarobot-oss/datarobot-agent-tester (2 stars, last pushed 26d ago), licensed Apache-2.0. It adds 691 tokens to every session, about $0.0035 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.

Building agents Testing

next.js AGENTS.md

AGENTS.md instructions for vercel/next.js, covering next.js development guide, codebase structure, monorepo overview, core package: packages/next and other important packages.

vercel/next.js · 7,296 tokens

codex AGENTS.md

AGENTS.md instructions for openai/codex, covering rust/codex-rs, the codex-core crate, code review rules, crate api surface and model visible context.

openai/codex · 5,153 tokens

vscode buildNext.instructions.md

Working notes and architecture documentation for the new esbuild-based build system in build/next. Use when making changes to the new build pipeline (transpile/bundle commands, NLS plugin, source-map handling, resource copying, or self-hosting watch tasks).

microsoft/vscode · 6,785 tokens

datarobot-agent-tester: Instructions file for Codex