terminal-bench-science: Skill for Claude Code

Review a benchmark task PR end-to-end. This skill downloads all artifacts, analyzes the task against the rubric, launches harbor view, and enters interactive review mode.

Prerequisites: gh CLI (authenticated), harbor CLI, jq. Run this from within your local clone of the benchmark repo.

Phase 0: Parse PR URL and Detect Repo

Parse the PR URL from $ARGUMENTS (strip trailing slash and any anchor/path suffixes after the PR number). Extract OWNER, REPO, PR_NUM.

Detect the local repo context from the current working directory.

Run a single Bash call:

# Parse PR URL — strip trailing slash and anything after pull/NN
PR_URL=$(echo "$ARGUMENTS" | sed 's|/changes.*||; s|/files.*||; s|/commits.*||; s|/*$||')
OWNER=$(echo "$PR_URL" | sed 's|https://github.com/\([^/]*\)/.*|\1|')
REPO_NAME=$(echo "$PR_URL" | sed 's|https://github.com/[^/]*/\([^/]*\)/.*|\1|')
PR_NUM=$(echo "$PR_URL" | sed 's|.*/pull/\([0-9]*\).*|\1|')
FULL_REPO="$OWNER/$REPO_NAME"

# Detect local repo
REPO_ROOT=$(git rev-parse --show-toplevel)
LOCAL_REPO=$(git remote get-url origin | sed 's|.*github.com[:/]\(.*\)\.git$|\1|; s|.*github.com[:/]\(.*\)$|\1|')

echo "PR: $FULL_REPO#$PR_NUM"
echo "Local repo: $LOCAL_REPO (at $REPO_ROOT)"

# Get PR metadata
gh api "repos/$FULL_REPO/pulls/$PR_NUM" --jq '{title: .title, headRef: .head.ref, headSha: .head.sha, baseRef: .base.ref, state: .state}'

# Detect task name from changed files
gh api "repos/$FULL_REPO/pulls/$PR_NUM/files" --paginate --jq '.[].filename' | grep '^tasks/' | head -20

From the files list, extract the task name (the directory under tasks/). Set TASK_NAME and REVIEW_DIR=$REPO_ROOT/../review-$TASK_NAME. Also save PR_TITLE from the metadata call above.

Save all these variables — you will need them throughout.

Every artifact this review produces must be identifiable on sight, because several reviews are usually open at once and their worktrees, viewers, and summaries all look alike. Carry #$PR_NUM — $PR_TITLE into the review summary's H1, the opening line of the summary you present to the user in Phase 11, and any top-level PR comment drafted in Phase 12. Never identify a review by task name alone: the same task is re-reviewed across follow-up PRs (a fix PR and its original merge share a task name and nothing else).

In the summary's H1, make the PR reference a clickable markdown link to $PR_URL — the summary is opened in a markdown viewer detached from this session, so the link is the only way back to the PR from the artifact. Keep the link text to the PR #<n> token so the title still reads as a title:

# Task Review: <task-name> — [PR #<pr-num>](<pr-url>): <pr-title>