Instructions file
Instructions for ihsaan-ullah/auto-codabench, covering claude.md, what this repo is, commands, keyless cli paths (work with no claude auth at all) and architecture.
2 1mo ago A 2,697 tokens
original MIT
Skill Claude CodeCodex
Phase 2 of an AutoCodabench session — read the locked implementationplan.md, write a complete Codabench bundle, then SELF-VALIDATE end-to-end by (a) running the bundle's own baseline submission through the scoring pipeline and (b) executing the starting-kit notebook inside the bundle's Docker image. Iterates on…
2 1mo ago A 122 tokens
original MIT
Skill Claude CodeCodex
Adapt one external submission's code to match a previously-built bundle's interface AND the libraries its Docker image ships, then run it through the bundle's scoring pipeline. Iterates on runtime errors (missing packages, API breaks, wrapper-shape mismatches) and returns parsed scores + full logs. Used by the…
2 1mo ago A 124 tokens
original MIT
Skill Claude CodeCodex
Technical schema and conventions for a Codabench competition bundle (competition.yaml, pages, phases, scoring program, ingestion program, scores.json↔leaderboard mapping, zip layout). Use when generating any file inside a bundle.
2 1mo ago A 51 tokens
original MIT
Skill Claude CodeCodex
Best practices for designing an AI competition — task framing, metrics, datasets, baselines, leaderboards, anti-cheating. Use when proposing or critiquing a competition idea.
2 1mo ago A 39 tokens
original MIT
Skill Claude CodeCodex
Phase 1 of an AutoCodabench session — through a short, citation-grounded roadmap conversation, produce one implementationplan.md covering all 7 design sections of a Codabench competition. The plan must be CONCRETE enough that Phase 2 can package a working Codabench bundle from it directly (sklearn-class baselines…
2 1mo ago A 119 tokens
original MIT