Tracely is a CI/CD system for AI agents that turns failed production traces into replayable regression tests. Development teams use it to detect and group agent failures, run the resulting cases on pull requests, and block changes that reproduce those failures. The catalogue entries provide skills for operating this trace-based testing and observability workflow.
Borrowing it
Nothing to install: this file belongs to Jwuthri/Tracely-ai. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.
curl -O https://raw.githubusercontent.com/Jwuthri/Tracely-ai/master/CLAUDE.mdgit clone --depth 1 https://github.com/Jwuthri/Tracely-aiWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/instructions/jwuthri/tracely-ai/claude-md)<a href="https://agentmods.dev/instructions/jwuthri/tracely-ai/claude-md"><img src="https://agentmods.dev/badge/instructions/jwuthri/tracely-ai/claude-md.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.03196 | $0.03196 |
| Opus 5 | $0.01598 | $0.01598 |
| Sonnet 5 | $0.00639 | $0.00639 |
| Haiku 4.5 | $0.00320 | $0.00320 |
Grade A, and why
Tracely-ai CLAUDE.md scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 98 lines — stays where its author put it; the contents beside it link to each section on GitHub.
CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
Tracely is trace-native CI/CD for AI agents: production trace → failure detection → regression test → CI/CD gate. The trace is the source of truth; evals, clusters, cases, gates and trends are all derived from it. There are no hand-authored datasets.
A uv workspace (backend, workers, sdk) plus a pnpm Next.js app (frontend) and a Nextra docs site (docs).
Commands
make install # uv sync --all-packages --all-extras + pnpm install
make infra-up # clickhouse, postgres, redis, minio (docker)
make migrate # ClickHouse DDL runner + Alembic (Postgres)
make seed # default project + ingest key `tracely_dev_key`
make backend / workers / frontend # three terminals: FastAPI :8000 · Celery · next dev
make demo # populate the whole product (traces, clusters, cases, gates)
Tests — no infra required, ~6s:
uv run pytest -q backend/tests sdk/tests # what CI runs
uv run pytest -q backend/tests/test_gate_eval.py::test_name -x # single test
uv run ruff check . && uv run ruff format .
cd frontend && pnpm test # vitest; pnpm test:watch; pnpm build type-checks (tsc) + lints
Alembic: cd backend && uv run alembic revision -m "…" / alembic upgrade head. ClickHouse migrations are *.up.sql files in backend/tracely/infrastructure/clickhouse/ddl/ applied by python -m tracely.infrastructure.clickhouse.migrations.
Whole stack in Docker: docker compose up -d --build --wait → UI on :3001, backend on :8000 (remap with TRACELY_WEB_PORT / TRACELY_BACKEND_PORT). make frontend runs plain next dev (:3000) — use cd frontend && pnpm dev -p 3001 to match Docker.
Architecture
Write path (deliberately mirrors Langfuse, reimplemented in Python):
SDK/OTLP → POST /v1/traces → S3 blob (durable FIRST) → Redis/Celery
→ worker: otel/ mapping → registry upsert → ClickHouse events
→ evaluate_run_task (countdown=4, debounces late spans) → scores + structural clustering
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 2d ago Changed · -10 tokens per session 759d770eb9fd
- 8d ago First seen · 98 lines · 3,206 tokens per session scan A 06fbcef57539
Tracely-ai CLAUDE.md is an instructions file published in the GitHub repository Jwuthri/Tracely-ai (1,215 stars, last pushed today), licensed MIT. It adds 3,196 tokens to every session, about $0.0160 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other instructions, from other repositories
codex AGENTS.md
AGENTS.md instructions for openai/codex, covering rust/codex-rs, the codex-core crate, code review rules, crate api surface and model visible context.
vscode buildNext.instructions.md
Working notes and architecture documentation for the new esbuild-based build system in build/next. Use when making changes to the new build pipeline (transpile/bundle commands, NLS plugin, source-map handling, resource copying, or self-hosting watch tasks).
next.js AGENTS.md
AGENTS.md instructions for vercel/next.js, covering next.js development guide, codebase structure, monorepo overview, core package: packages/next and other important packages.
langchain AGENTS.md
AGENTS.md instructions for langchain-ai/langchain, covering global development guidelines for the langchain monorepo, corridor security analysis, project architecture and context, monorepo structure and development tools & commands.
vscode oss-third-party-notices.instructions.md
Instructions for microsoft/vscode, covering vs code oss third-party-notices pipeline, architecture, pipeline flow in ci, applying the notice (cutover) and fallback chain (never fail the build).
spec-kit AGENTS.md
AGENTS.md instructions for github/spec-kit, covering agents.md, about spec kit and specify, quickstart — add a new integration in 5 steps, integration architecture and integrationmanifest — file tracking.