e2e-template-testing

Context

Squad's coordinator prompt (squad.agent.md) and agent charters (e.g. scribe-charter.md) are shipped as templates in .squad-templates/. Changes to these files affect how every squad session behaves — but unit tests can't catch prompt-level regressions because the prompts are interpreted by an LLM at runtime.

This skill describes how to validate template changes end-to-end by running real squad sessions against a locally-built CLI that includes your modified templates.

When To Use

• You changed .squad-templates/squad.agent.md (coordinator prompt)
• You changed .squad-templates/scribe-charter.md or other agent charters
• You changed .squad-templates/notes-protocol.md or helper scripts
• You added new conditional blocks (e.g. state-backend-aware spawn templates)
• You modified the init scaffolding that writes templates to target repos

Prerequisites

• Node.js ≥20, npm ≥10
• Git CLI
• GitHub Copilot CLI (copilot or ghcs) installed
• A local clone of the squad repo on your feature branch

Workflow

Step 0 — Post initial tracking comment (FIRST action — before anything else)

If PR_NUMBER and REPO are both set, the absolute first thing you do — before fast-fail checks, before building, before creating any repos — is post the initial tracking comment with all steps marked as :hourglass_flowing_sand: Pending.

This gives reviewers immediate visibility that a run is in progress and what to expect.

$runStart = Get-Date
$body = @"
## E2E Progress - PR $env:PR_NUMBER

| Step | Status | Started | Duration |
|---|---|---|---|
| 1. Fast-fail checks (build :cd: link :cd: ``squad version``) | :hourglass_flowing_sand: Pending | --:-- | -- |
| 2. Create test repo(s) | :hourglass_flowing_sand: Pending | --:-- | -- |
| 3. ``squad init`` + file verification | :hourglass_flowing_sand: Pending | --:-- | -- |
| 4. Run sessions | :hourglass_flowing_sand: Pending | --:-- | -- |
| 5. Verify outcomes | :hourglass_flowing_sand: Pending | --:-- | -- |
| 6. Record verdicts + post final comment | :hourglass_flowing_sand: Pending | --:-- | -- |

| Symbol | Meaning |
|---|---|
| :hourglass_flowing_sand: | Not started |
| :arrows_counterclockwise: | Running |
| :white_check_mark: | Passed |
| :x: | Failed |
| :warning: | Passed with caveats |

*Run started: $($runStart.ToString('HH:mm')) — all steps pending*
"@

$tmpFile = [System.IO.Path]::GetTempFileName()
$utf8NoBom = New-Object System.Text.UTF8Encoding $false
[System.IO.File]::WriteAllText($tmpFile, $body, $utf8NoBom)
$response = gh api "repos/$env:REPO/issues/$env:PR_NUMBER/comments" --method POST --field "body=@$tmpFile" | ConvertFrom-Json
$env:COMMENT_ID = $response.id
Remove-Item $tmpFile -Force
Write-Host "Progress comment posted — ID: $($response.id)"