Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add datadog-labs/agent-skills --skill triage-flaky-testgit clone --depth 1 https://github.com/datadog-labs/agent-skillsWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/datadog-labs/agent-skills/triage-flaky-test)<a href="https://agentmods.dev/skills/datadog-labs/agent-skills/triage-flaky-test"><img src="https://agentmods.dev/badge/skills/datadog-labs/agent-skills/triage-flaky-test/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/datadog-labs/agent-skills/triage-flaky-test"><img src="https://agentmods.dev/badge/skills/datadog-labs/agent-skills/triage-flaky-test.svg" alt="Reviewed on agentmods" width="80" height="20"></a>- NVIDIA SkillSpector pass
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00032 | $0.02909 |
| Opus 5 | $0.00016 | $0.01455 |
| Sonnet 5 | $0.00006 | $0.00582 |
| Haiku 4.5 | $0.00003 | $0.00291 |
Grade A, and why
triage-flaky-test scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 11d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
Copies of this mod
1 near-identical copy found in the catalogue:
- triage-flaky-test — 100% identical, 0 lines differ
How it starts
The opening of the file, as written. The whole thing — 257 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Triage Flaky Test
One-line summary: Investigate a specific flaky test — get history, failure pattern, and category, then recommend fix, quarantine, or escalate.
Requires: dd-pup skill (pup CLI installed and authenticated).
Backend
Detection — At the start of every invocation, before taking any action, determine which backend to use:
- If the user passed
--backend pupanywhere → use pup mode immediately. Skip steps 2–4. - Check whether
get_datadog_flaky_testsappears in your available tools. - If present → use MCP mode throughout. Call tools exactly as named in this skill's workflow sections.
- If absent → check whether
pupis executable: runpup --versionvia Bash. If the command exits successfully (exit code 0), pup is available. - If pup responds → use pup mode throughout. Translate every tool call using the Tool Reference appendix at the bottom of this file.
- If neither is available → stop and tell the user:
"Neither the Datadog MCP server nor the pup CLI is available. Connect the MCP server or install pup (
brew install datadog-labs/pack/pup)."
pup invocation rules:
- Invoke via Bash. pup always outputs JSON — parse directly.
- Repository IDs passed to pup must be fully lowercase (the API rejects mixed-case).
- Sort values starting with
-require=syntax:--sort="-last_flaked"(not--sort "-last_flaked"). - If pup returns a 401/403, tell the user to run
pup auth refreshorpup auth login.
Input
| Parameter | Description |
|---|---|
| Test name | Fully qualified test name (e.g. TestMyFunc or com.example.MyTest) |
| Repository | Lowercase, no-schema URL (e.g. github.com/org/repo). Derive from git remote get-url origin if not provided. |
Workflow
STEP 0 — Parse Input
Derive repository ID from git if not provided:
git remote get-url origin
# Strip protocol and trailing .git, then lowercase the result
# e.g. https://github.com/DataDog/my-repo.git → github.com/datadog/my-repo
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 11d ago First seen · 257 lines · 32 tokens per session scan A 224a9f648b9d
triage-flaky-test is a skill published in the GitHub repository datadog-labs/agent-skills (168 stars, last pushed 15d ago), licensed MIT. It adds 32 tokens to every session and 2,909 once invoked, about $0.0002 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
systematic-debugging
4-phase root cause debugging: understand bugs before fixing.
langsmith-observability
LLM observability platform for tracing, evaluation, and monitoring. Use when debugging LLM applications, evaluating model outputs against datasets, monitoring production systems, or building systematic testing pipelines for AI applications.
experimental-code-coverage-local-debugger
Runs code coverage locally via Universal Test Runner (UTR) or helper scripts, mimicking LUCI trybots. Activate when CQ tryjobs fail or underreport coverage, to test local GN/recipe repairs before uploading, or to debug hermetic crashes.
adversarial-reviewer
Adversarial code review that assumes bugs exist and hunts for them. Use when asked to review code, find bugs, audit for correctness, stress-test a PR, or when someone says "tear this apart" or "what's wrong with this". Give no benefit of the doubt — every line is guilty until proven innocent.
cli-e2e
Write, modify, or debug Docker-based Composio CLI end-to-end tests under ts/e2e-tests/cli, including binary invocation, fixture isolation, output assertions, and package manifests. Use for CLI E2E test suites only; use cli-command for CLI source implementation.
work
Deliver one maintainer-approved EmDash issue, choosing the bug-fix path for a defect and the direct implementation path for an enhancement or task.