AutoTrainess AGENTS.md

AGENTS.md

Objective

• Improve the target task or benchmark through real training and evaluation.
• Base important decisions on verifiable evidence.

Hard Constraints

• APIs may be used only for benchmark evaluation, never for data construction or any other non-evaluation purpose.
• Do not use benchmark examples, the values of any fields or components of benchmark examples, or any data whose overlap with the benchmark cannot be ruled out for training or to create training data.
• Fine-tune only from the exact target base model provided by the task or checkpoints you fine-tuned from it; do not use an instruction-tuned, chat-tuned, larger, or different model as the training start point, merge source, fallback, or final submission.
• Never tune, search, or modify generation config to improve benchmark scores.
• If CUDA_VISIBLE_DEVICES is set, it defines the only GPU devices assigned to this run; do not override it or use any GPU outside that set.
• A run being blocked by environment, permissions, dependencies, shared resources, or turn-level execution limits does not mean the task is complete.

Experiment Guidance

• Prefer the simplest valid approach first.
• Record the concrete reason for each failure, improvement, and strategy change.
• State which stage you are currently in.
• Do not spend iterations repeatedly tuning low-level settings unless there is clear evidence that doing so targets the current bottleneck.

Stage Rules

• Stages are sequential and mandatory.
• If the current stage fails because of an engineering or environment issue, fix it and retry that stage.
• If the current stage has been explored enough to justify moving on, enter the next stage instead of ending the task.
• If no explicit target is provided, do not decide on your own that the task is complete.

Iteration Execution

• For Stage 2/3, each stage is executed through one or more full iterations.
• Skills define how to execute one iteration locally. If a skill's default options conflict with the current stage rules, follow the stage rules in this file.
• A full iteration should use the skill chain: iteration_plan -> data -> train -> eval -> log.
• iteration_plan defines the goal and planned changes for the current iteration.
• data handles data selection, construction, and validation before training.
• train runs training and exports final_model/.
• eval runs the benchmark's real evaluation on final_model/.
• log records the completed iteration and its outcome.