Synaptic-Tuner AGENTS.md

Project Agent Notes

This repository has a few cloud-training constraints that are easy to relearn the hard way.

Fine-Tuning Workflow Discipline

• For any task in this repo, begin by loading the most relevant canonical skill from .skills/. For fine-tuning, cloud training, evaluation, experiment-loop, checkpoint-eval, model-selection, or dataset-publishing work, that starting point is usually the fine-tuning skill.
• .skills/ is the canonical skill source for this repo. .agents/skills and .claude/skills are synced copies and must match it exactly.
• After changing canonical skills, run python3 .skills/scripts/sync_skill_trees.py and verify with python3 .skills/scripts/sync_skill_trees.py --check.
• Before building a new script, helper, or one-off workflow to run something, first check whether the needed command, script, CLI surface, or skill guidance already exists in the repo.
• Do not create throwaway scripts just to get a task done if an existing script, CLI, or skill can be used or extended.
• If the capability does not exist, the next step is not an ad hoc workaround. Update the relevant skill and add the proper checked-in script/CLI workflow so the new capability is reusable.
• Prefer repo CLIs and checked-in scripts over manual bucket/API probing whenever those surfaces exist.

Config-First Generation Discipline

• This repo is format-agnostic. Do not treat the current tool wrapper, CLI shape, or toy dataset format as a runtime truth.
• For generation and evaluation tasks, do not change runtime code to support one user's current tool schema, wrapper, commands, examples, or dataset shape. Use config, scenario YAML, rubric YAML, schema files, or checked-in declarative config instead.
• Only change parser, executor, judge, evaluator, or generation code when the project is intentionally adding a reusable runtime capability that cannot be expressed by existing config surfaces. If that is necessary, stop first and explain why config is insufficient.
• Do not hardcode any current dataset/example format into parser, executor, judge, evaluator, or generation code.
• Tool-call shapes, wrapper names, context fields, command examples, and dataset-specific assumptions must live in config or scenario/rubric YAML, not in code.
• If a generation/eval bug appears to be specific to the current CLI/tool wrapper, the first fix path is config, rubric, or scenario work, not runtime code changes.
• Generic runtime code may validate or transport configured formats, but it must not assume one specific wrapper such as useTools, one specific field set, or one specific command structure unless that behavior is itself config-driven.
• When using environment-backed validation during generation, environment/runtime errors must be surfaced into the judge/improver inputs as structured context so the model can correct the response. Do not rely on ad hoc parser/executor repairs as the primary fix path.
• If a process requirement is important enough to affect how generation/eval work is done repeatedly, encode it in AGENTS.md, the canonical skill under .skills/, and the relevant checked-in config/docs before continuing.
• For new unsteered local batch generation, prefer batch-generate --engine vllm. Set VLLM_BATCH_INVARIANT=1 before process startup and pin the exact vLLM version, documented minimum CUDA compute capability, model and tokenizer revisions, dtype, tensor parallel size, scheduler limits, structured-output backend, model context limit, multimodal limits, decode settings, and JSON Schema when formatting is incidental. Keep generated rows and token artifacts in the consuming project's private output directory.