TruLens is an open-source system for tracing and evaluating LLM applications and AI agents. It records each step's inputs, outputs, latency, tokens, and cost, then uses evaluations to find failures and compare application versions.
Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add truera/trulens --skill eddgit clone --depth 1 https://github.com/truera/trulensWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/truera/trulens/edd)<a href="https://agentmods.dev/skills/truera/trulens/edd"><img src="https://agentmods.dev/badge/skills/truera/trulens/edd/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/truera/trulens/edd"><img src="https://agentmods.dev/badge/skills/truera/trulens/edd.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00063 | $0.01013 |
| Opus 5.5 | $0.00025 | $0.00405 |
| Sonnet 5 | $0.00013 | $0.00203 |
| Haiku 4.5 | $0.00006 | $0.00101 |
Grade A, and why
edd scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured today.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 106 lines — stays where its author put it; the contents beside it link to each section on GitHub.
EDD loop for rag-loop with TruLens (aircraft systems)
Orchestration for this repo's eval loop: which command to run, in what order, and what's automated
vs. what needs judgment. For TruLens evaluation setup, instrumentation, or feedback functions, reference
TruLens core patterns (Feedback, Selector, SpanAttributes).
Stack: LangGraph orchestrates retrieve → generate; retrieval embeddings are OpenAI
(text-embedding-3-small, requires OPENAI_API_KEY); generation uses OpenAI
(gpt-5.6-luna); TruLens feedback judges use the RAG Triad with OpenAI
(gpt-4.1-nano);
vector database is Qdrant (QDRANT_URL/QDRANT_API_KEY). Traces and evaluations are recorded via
TruLens OpenTelemetry instrumentation and saved in TruLens database.
Two fundamental rules:
- Never invent an
expected_output. Every golden's expected answer must be lifted from the actual PDF text, not written from general knowledge of aircraft systems. - Grade outcomes, not paths. A judge scores whether the final answer is relevant, grounded in the retrieved context, and directly answers the question.
Preconditions
Everything except reading/editing files needs .env populated:
OPENAI_API_KEYQDRANT_URLQDRANT_API_KEY
rag-loop run also requires git tracking to commit accepted changes and revert rejected ones.
What's automated vs. what's your job
rag-loop run <name> (src/rag_loop/loop.py: run_iteration) automates the mechanical steps:
- Runs the experiment through the RAG pipeline
- Evaluates with TruLens feedback functions (
answer_relevance,groundedness,context_relevance) - Computes
clean_negative_recallfor unanswerable held-out items - Persists scores and COT reasons to
evals/results/<name>.json - Compares against
acceptance.compare_againstonacceptance.required_metrics - Writes the decision back to
evals/experiments.json - Commits
src/(keep) or reverts via git (reject)
Your job as an agent: read the failures, form a hypothesis, and write the next single-variable change. The iterative cycle:
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- today First seen · 106 lines · 63 tokens per session scan A a05d2d0c9e67
edd is a skill published in the GitHub repository truera/trulens (3,571 stars, last pushed yesterday), licensed MIT. It adds 63 tokens to every session and 1,013 once invoked, about $0.0003 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-24.
Other skills, from other repositories
evaluators
Author or refine a Phoenix evaluator — code or LLM-as-a-judge — that scores a run's output. Trigger when the user wants to create a new evaluator, improve an existing one's logic or rubric, choose labels, or decide what to measure on a dataset or experiment. Do NOT trigger on: (1) manual prompt drafting (use…
playground
Author, edit, or iterate on prompts in the Phoenix prompt playground, including running experiments over a dataset. Load before any playground ui. operation call, including single-shot prompt rewrites.
datasets
Understand what a Phoenix dataset is and reason well about its examples, outputs, splits, and how it feeds evaluators and experiments. Load this whenever a dataset is in view or the user asks what a dataset is, how splits work, what an output "means", or how datasets relate to experiments and evals. This skill governs…
experiments
Run, read, and compare dataset-backed experiments to find evidence that a prompt or pipeline is improving. Trigger when the user wants to iterate over a dataset with experiments, compare experiment runs, read experiment quality/latency/cost, or decide whether a change actually helped. Running a prompt over a dataset…
phoenix-client-development
Development guide for the @arizeai/phoenix-client TypeScript SDK — run and resume experiments, manage OpenTelemetry tracer providers with stack-based attach/detach, and write vitest unit and integration tests. Use when adding features to phoenix-client, debugging experiment lifecycle or provider cleanup, modifying…
b02-skills-main--datascience
🤖 Data Science Engineering Skills — TDD and planning skills for ML pipelines, data APIs and analytical tooling. Derived from skills-main (mattpocock/skills). Data pipelines, model training, evaluation, MLOps.