arbor-agent-plugins-hitl-budget

Arbor Plugins, HITL, And Budget

Use this when run behavior is modified by configuration rather than by code.

Plugin Contract

A plugin is a YAML domain adapter. It can define:

• Prompt injections: meta_preamble_inject, meta_init_inject, meta_ideate_inject, meta_decide_inject, sub_preamble_inject, sub_workflow_inject.
• eval_contract: prefilled metadata such as metric_direction, eval_cmd, submission_path, and sample_submission_path.
• protected_paths: globs a branch must not modify.
• required_outputs: files that must exist for merge.
• config_overrides and named profiles.
• lifecycle_hooks: on_workspace_setup, after_executor, on_finalize.
• convergence: early stop policy.

Plugin precedence:

defaults < plugin.config_overrides < active profile < project YAML < CLI

MLE/Kaggle Mode

The built-in mle_kaggle plugin is performance-first:

• Novelty has zero value; only metric improvement matters.
• skills_enabled may be disabled for strict idea drafting because parameter tuning, prompt edits, feature engineering, ensembling, and scaling can be valid.
• A valid submission.csv is a first-class artifact.
• Data/eval directories are protected.
• Merge requires required outputs and verified score.

Priority:

1. Produce and evaluate a valid baseline submission.
2. Establish robust validation.
3. Explore diverse approach families.
4. Refine promising families.
5. Ensemble/blend diverse winners.
6. Reserve finalization time for best submission recovery.

Budget Policy

Default Arbor behavior favors long real experiments. Do not invent staged budgets unless configured.

If budget_policy.stages exists, use configured stages such as smoke, pilot, and full with RunTraining(budget_stage=...).

If require_cost_estimate is true, every idea needs:

• estimated wall time;
• minimal smoke test;
• promotion gate from cheap to expensive fidelity.

Do not launch a new executor when remaining budget cannot cover the executor plus finalization buffer.