AgentKernelArena: Instructions file for Codex

AGENTS.md

Purpose

AgentKernelArena is a controlled environment for evaluating GPU-kernel agents. Every change must preserve reproducibility, benchmark integrity, meaningful correctness checks, and fair baseline-versus-candidate comparison.

Read README.md and CONTRIBUTING.md before making broad changes. Use repository-relative paths in code, configuration, docs, and instructions; never embed paths from a local checkout.

Repository map and sources of truth

• main.py: run orchestration, resume, and parallel scheduling.
• src/: shared prompting, workspace setup, evaluation, scoring, and reporting.
• agents/: isolated agent integrations and their own configuration.
• tasks/: self-contained kernel tasks and task-local runners.
• example_configs/: run-level agent, task, and GPU selections.
• src/tools/perf/: canonical benchmark helpers materialized into workspaces.
• src/eval_tools/: evaluator-side optional analysis-tool control plane.
• tests/: framework, integration, and regression tests.

When behavior and prose disagree, inspect the implementation. In particular:

• Agent identifiers and loading: src/module_registration.py
• Task definition, schema, and authoring contract: docs/how-to/add-task.md
• Task-specific declarations: the task's config.yaml
• Evaluation and scoring: src/evaluator.py, src/performance.py, src/score.py
• Harness protection: src/harness_guard.py
• Benchmark helpers: src/tools/perf/
• Evaluation-tool policy: docs/how-to/use-evaluation-tools.md

Update relevant docs in the same change when behavior or public configuration changes. Avoid copying volatile model IDs, image tags, or complete registries into new guidance; link to their source of truth instead.

File placement and repository cleanliness

• Before adding a file, identify its lasting purpose and the existing directory that owns it. Prefer updating the relevant source, test, configuration, or documentation in place.
• Keep temporary plans, scratch notes, session handoffs, PR status summaries, one-off job scripts, and validation campaign records outside the repository unless the user explicitly requests that they be versioned. Record validation results for the change in the commit or PR description; keep reusable usage instructions in the existing documentation.
• Keep example_configs/ for maintained, reusable run selections. PR-specific task lists and retry-round configs are local experiment inputs. When the Docker/Slurm runner needs a config inside the checkout, use an ignored config_*.yaml file and confirm it is untracked before committing.
• Task directories contain task contracts, sources, inputs, harnesses, required instructions, licenses, and source attribution. Do not add draft trackers or campaign summaries beside the tasks. Preserve required attribution and task-local documentation when removing redundant summaries.
• Write experiment output only to its configured output locations and keep it untracked. Do not copy generated workspaces, logs, or report bundles into source or documentation directories to make them part of a commit. This includes task build/ directories, *_report.json, validation_report*, task_result.*, and run summaries. Preserve evidence with its source/config identity and runtime details outside the versioned task package.
• Before every commit, follow the contribution checklist. Inspect untracked files and both unstaged and staged changes. Stage explicit paths; review every added file for a lasting purpose and every deletion for references and required attribution. .gitignore does not untrack files already committed, so inspect the staged file list even when ignore rules exist.
• Before finishing, remove disposable scratch files you created inside the repository. Preserve pre-existing user files and experiment artifacts; the artifact-preservation rules below still apply. Do not add ignore rules merely to hide misplaced source files or documentation.