post-deploy-verification

Post-deploy verification

A deploy that went green tells you the process started. It says nothing about whether the change it shipped actually works, or whether it broke something else that happened to keep running. Verification is what closes that gap, and it happens every time — not only when something looks wrong.

1. Procedure — exact calls

1. get_release({ task_id }) → read deployed_at, component, checks (health samples, smoke results, new_errors, notes, early_stop), profile.auto_rollback, and tasks[] — every task this release covers, not only the one you were woken on.
2. get_environment only if you need to confirm nothing is bound (checks.notes already says so in most cases).
3. list_runtime_errors({ component, since: "<window that starts well before deployed_at>" }) — see "Reading error groups" below for the window and how to judge new.
4. query_runtime_logs({ component, since: deployed_at, min_severity: "error" }); re-query with text: set to a route or module the release changed when the error list alone is not enough.
5. run_smoke_checks({ task_id }) when the stored evidence is older than ~30 minutes, or when early_stop itself came from a smoke failure and you need a fresh read before deciding.
6. One read-only acceptance check per observable criterion of every task in release.tasks (list_acceptance_criteria per task) — a release with several tasks is verified in full, not just the card that woke you.

2. Reading error groups

list_runtime_errors' new: true is accurate on a provider with native grouping (gcloud). On a provider without one (Vercel, AWS — fallback fingerprint grouping), it only means "this group's first occurrence fell in the back half of whatever window you queried" — querying since: deployed_at puts a regression that starts right after the deploy in the window's front half, so it can read new: false and never show up as a concern.

• Judge a group by its own first_seen against release.deployed_at, never by new alone.
• Read with a window that starts well before deployed_at — roughly twice the time elapsed since the deploy, or at least the soak period — so an occurrence from before the deploy is visible and can be told apart from one that started with it.
• Before rolling back on one low-count group, re-read with since: "1d". If the same message recurs before deployed_at, it is chronic, not a regression. Once you have confirmed a group is chronic and unrelated to any release, save_memory (project scope) a one-line note of its fingerprint message ("redis timeout in apps/api recurs hourly; not a release signal") so a future run of this skill recognizes it without a full 1-day re-read.
• checks.new_errors (the sweeper's own field) can be empty on Vercel or AWS even when errors genuinely began with the deploy — your own read in this run is the check, not a formality that confirms what the sweeper already decided.