binlog-failure-analysis

Binlog Failure Analysis (reuse AzDO binlog, don't rebuild)

Analyze a failed AzDO PR build by reading the binlog that build already produced, not by re-running the local build (./build.sh -bl or equivalent) on a GitHub Actions runner. This is the AI-build-failure-analyst pattern: it makes the analyst run cheap enough to be turned on for every failing PR sync.

🛑 NEVER use gh pr review --approve or --request-changes. Only --comment is allowed. Approval and blocking are human-only actions.

Workflow: gate on check_run: completed → resolve PR + build id → download AzDO artifact → ask the binlog MCP server for the build overview, errors, and warnings (drilling into root-cause diagnostics or property-value tracing when the basics aren't enough) → group symptoms by root cause → post a single summary comment → optionally attach ≤10 inline suggestion blocks. The agent drives the analysis; tools provide the data.

When to use

• A GitHub PR check from app slug azure-pipelines (or the repo's AzDO check slug) transitions to failure.
• The pipeline publishes a binlog artifact (arcade-onboarded repos do this via eng/common/core-templates/steps/publish-logs.yml; arbitrary AzDO pipelines may not).
• You want a structured per-PR comment with grouped root causes and concrete suggestion blocks — not just a link to the raw log.

When NOT to use

• The PR has no AzDO check (GH-Actions-only repos) — use a rebuild-based skill instead, or wait for an AzDO migration.
• The pipeline doesn't publish a binlog artifact — there's nothing to reuse.
• You want pipeline-wide health analysis across many builds — use ci-analysis or pipeline-investigation.
• You want to investigate a non-build leg failure (test failures, Helix work item failures) — see helix-investigation.

Prerequisites

• binlog MCP server — Microsoft.AITools.BinlogMcp is wired in plugins/dotnet-dnceng/plugin.json under the namespace binlog. The same server is also published by dotnet/skills/dotnet-msbuild under the same binlog name, so installing both plugins resolves to a single MCP instance. This skill invokes the server's capabilities semantically (build overview, error list, warning list, root-cause diagnose, property-value trace, structured search, task details, tasks-in-target) — discover the exact tool names for your server version via tools/list. With Microsoft.AITools.BinlogMcp, typical names start with binlog_ (e.g., binlog_overview, binlog_errors); the server exposes ~29 tools total.
• curl for the AzDO REST artifact download.
• jq for parsing the AzDO artifacts JSON and the GitHub check-runs payload (Step 2 + references/azdo-artifact-fetch.md).
• unzip for extracting the artifact.
• Azure DevOps access:
  • Anonymous read for dev.azure.com/dnceng-public/public — no token needed.
  • For devdiv/DevDiv or dnceng/internal: a federated identity (azure/login@v2) or azureauth ado token. See references/azdo-artifact-fetch.md.
• gh CLI for posting the summary comment + inline review comments.