autocodabench-reformat-and-run

AutoCodabench — Reformat & Run

You are given:

• a Codabench bundle_dir (already runtime-validated by an earlier autocodabench-implement invocation: its baseline runs cleanly, its notebook executes cleanly),
• a submission_dir containing one external submission's code (a ground-truth sub_N/submission/ directory),
• an env_name (accepted for compatibility; ignored — execution is Docker-only),
• an out_dir where you must write the adapted submission, logs, and parsed score JSON.

Your job: adapt the submission so it runs against the bundle's interface and inside the env's libraries, run it through scoring, write the result. No comparison against any expected score — that's the orchestrator's job after you finish.

0. Hard rules

1. API adaptation only — never re-scoping. You may:

   • rename tf.keras.optimizers.legacy.Adam → tf.keras.optimizers.Adam,
   • swap from keras.preprocessing import X → from tensorflow.keras.preprocessing import X,
   • wrap an old predict(X) shape into the bundle's expected predict(X) → labels shape,
   • adapt the submission to the libraries the bundle's docker_image ships (execution is Docker-only; you cannot install packages at run time, and the platform installs nothing).

   You MUST NOT:

   • swap the model class to a smaller one because GPU isn't available,
   • change hyperparameters, loss, metric, seed, epoch count,
   • generate synthetic data,
   • read the bundle's reference_data labels and hard-code predictions against them. If you ever feel tempted, stop — that's leakage that invalidates the entire experiment.

2. No access to expected_result.json, plan, or input/report.pdf. Your inputs are the four arguments above. Any other file under the experiment run dir is off-limits.

   Runs execute inside a Linux container, so the image's own library defaults apply. If a thread-pool deadlock is suspected, pass single-thread overrides for the call via extra_env= on autocodabench_run_user_submission (e.g. {"OMP_NUM_THREADS": "1", "OPENBLAS_NUM_THREADS": "1"}); they are applied as -e flags before the container's python starts. Do NOT add os.environ.setdefault(...) to the submission code — by the time that line runs the library is already loaded.