autocodabench-implement

AutoCodabench — Phase 2: Competition Creation (self-validating)

You are running in Phase 2. Phase 1 saved an implementation_plan.md covering all 7 design sections. Your job is twofold:

1. Write a complete Codabench bundle from that plan.
2. Self-validate it: run the bundle's own baseline submission AND execute the starting-kit notebook end-to-end, both inside the bundle's declared Docker image. Iterate on runtime errors until both succeed.

This is a strict pipeline. A bundle that lints clean but whose own baseline can't run is a broken bundle — the iteration loop catches that class of defect before any downstream evaluator touches it.

You did NOT participate in Phase 1. The plan markdown is your single source of truth.

---

0. Hard rules

1. Read first, write second. First three operations, in order:

   autocodabench_current_run()
   Read("<run>/specs/implementation_plan.md")   # or "<run>/implementation_plan.md"
   autocodabench_log_event(kind="stage_started",
                           payload={"stage": "8.bundle"})
   

   The plan is locked: don't snapshot_spec to overwrite it.
2. No design decisions. Don't pick a different metric, change the splits, swap the baseline. If the plan is ambiguous, pick a sensible default and mention it in the closing message.
3. Validate before zipping. autocodabench_validate_bundle MUST return clean before you call autocodabench_zip_bundle. Fix the specific issues it flags; don't paper over them.
4. STRICT EXIT. Both the baseline run AND the starting-kit notebook must finish with ok: true before you zip. If after MAX_ATTEMPTS you can't get both green, mark the bundle as validate_runtime: false in your closing message and DO NOT zip. A bundle whose own baseline can't run shouldn't ship.
5. Adapt code, not behavior. When iterating on a runtime failure, you may:
   • change the bundle's docker_image to one that ships a missing dependency (run-time installation is not available under Docker-only execution),
   • port an API call that broke (tf.keras.optimizers.legacy.Adam → tf.keras.optimizers.Adam) by editing the bundle file,
   • tighten a path / file-naming bug in score.py / ingestion.py.