trapstreet-solution-scaffold

trapstreet-solution-scaffold

Scaffolds a trap-cli solution directory for a task on trapstreet.run, and helps diagnose the real submission failures this ecosystem produces. Built from repeated real incidents, not theory -- every gotcha below actually happened.

tp missing, or tp auth status shows no valid pairing? Hand off to trapstreet-setup first -- everything below assumes tp already runs.

Ground rules

• Never read the task's expected/, judge.py, or grader.py to construct or embellish an answer -- only the IO-contract part of traptask.yaml (inputs/outputs shape) is fair game, and only as much of it as writing the adapter/solution.py actually requires.
• tp run/tp submit pause on two confirmation gates -- remote-source (about to execute code pulled from a repo) and unanchored (no git provenance, won't rank). Both are the user's call, never yours: explain the consequence in plain words, recommend an answer, let them decide (see "Gates and consent" below). Never pass --trust-remote/--allow-unanchored on your own initiative.
• Never submit to the public leaderboard without the user's explicit go-ahead on that specific submission -- a prior yes does not carry forward to the next run.

Before writing anything: interview

1. Which task, and where does it live? A local path (most common when the task repo is checked out nearby), a git+URL, or a trapstreet.run task page (https://trapstreet.run/tasks/<slug>). For the trapstreet.run case, resolve it via the public API rather than fetching the page itself -- faster, more reliable, and doesn't depend on page layout or content-blob parsing:

   curl -s https://trapstreet.run/api/tasks/<slug> | python3 -m json.tool
   # -> task.latest.{repo_url, commit_sha, repo_path}
   

   Assemble git+<repo_url>@<commit_sha>#subdirectory=<repo_path> yourself from those three fields -- don't scrape the task's web page for this, and don't ask the user to copy a git URL off it by hand. If local, you'll need the exact relative path from wherever each trap.yaml ends up -- get this right per-file (see Layout below), since a wrong relative depth is a common mistake.
2. Which of the three starting points is this? (see below) -- from scratch, wrapping an existing solution.py, or adapting an existing external project. This determines whether the scaffold script writes a template solution.py for you or leaves that part to you.
3. One model, or several to compare? This determines the layout (see below). If several: which providers/models exactly?
4. API keys, then model IDs. Before verifying anything, confirm the user actually has an API key for each provider in play -- ask directly if unsure; don't assume .env has real values just because .env.example exists. A missing key isn't obvious until solution.py crashes on a bare KeyError at runtime, so catch it here instead of there. If it's missing, point to where to get one and pause until the user has it:
   • Anthropic: https://console.anthropic.com/settings/keys
   • OpenRouter: https://openrouter.ai/keys
   • Any other provider: its own dashboard/account settings page