canary-test-reporter

Canary Test Reporter

Turn a Playwright JSON results file into a human-readable Markdown report and/or a machine-readable JSON artifact. Designed to run after the Playwright step in CI (if: always()) and upload both files as job artifacts.

Invocation

# Markdown to stdout:
canary skills run canary-test-reporter -- --results test-results/results.json

# Markdown to file:
canary skills run canary-test-reporter -- \
  --results test-results/results.json \
  --markdown-out test-results/report.md

# JSON to file:
canary skills run canary-test-reporter -- \
  --results test-results/results.json \
  --json-out test-results/report.json

# Both at once (recommended for CI):
canary skills run canary-test-reporter -- \
  --results test-results/results.json \
  --markdown-out test-results/report.md \
  --json-out test-results/report.json

# Usage and the full flag list (exits 0):
canary skills run canary-test-reporter -- --help

Exit code: 1 when any test failed; 0 otherwise. Flaky tests and skipped tests never affect the exit code.

Output formats

Markdown

# Test Report

**2 failed** · **1 flaky** · **14 passed** · **1 skipped** · 18 tests · 12.4s

## Failed (2)

### suite > spec > test title
`tests/auth.spec.ts:42`

```
Expected: 401
Received: 200
```

## Flaky (1)

- `tests/search.spec.ts:17` — search > autocomplete > debounce

## Summary

| Status | Count |
| --- | --- |
| Passed | 14 |
| Failed | 2 |
| Flaky | 1 |
| Skipped | 1 |
| **Total** | **18** |

JSON

{
  "version": 1,
  "generated_at": "2026-07-13T20:07:00Z",
  "summary": {
    "total": 18,
    "passed": 14,
    "failed": 2,
    "flaky": 1,
    "skipped": 1,
    "duration_ms": 12400
  },
  "results": [
    {
      "title": "suite > spec > test",
      "status": "failed",
      "file": "tests/auth.spec.ts",
      "line": 42,
      "duration_ms": 1823,
      "error": "Expected: 401\nReceived: 200"
    }
  ]
}

The version field pins the contract for downstream tooling. results includes all tests so external tools can compute their own views.