TruLens is an open-source system for tracing and evaluating LLM applications and AI agents. It records each step's inputs, outputs, latency, tokens, and cost, then uses evaluations to find failures and compare application versions.
Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add truera/trulens --skill trulens-evaluation-setupgit clone --depth 1 https://github.com/truera/trulensWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/truera/trulens/trulens-evaluation-setup)<a href="https://agentmods.dev/skills/truera/trulens/trulens-evaluation-setup"><img src="https://agentmods.dev/badge/skills/truera/trulens/trulens-evaluation-setup.svg" alt="Measured on agentmods" height="20"></a>- NVIDIA SkillSpector pass
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00017 | $0.03080 |
| Opus 5 | $0.00009 | $0.01540 |
| Sonnet 5 | $0.00003 | $0.00616 |
| Haiku 4.5 | $0.00002 | $0.00308 |
Grade A, and why
trulens-evaluation-setup scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 8d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 467 lines — stays where its author put it; the contents beside it link to each section on GitHub.
TruLens Evaluation Setup
Configure feedback functions to evaluate your LLM app's quality.
Interactive Evaluation Selection
Before proceeding, let's determine the right evaluations for your app.
Question 1: What type of app are you building?
Option A: RAG (Retrieval-Augmented Generation)
- Your app retrieves documents/chunks from a knowledge base
- It generates responses grounded in the retrieved context
- Examples: Q&A systems, document search, knowledge assistants
→ Recommended: RAG Triad metrics
- Context Relevance
- Groundedness
- Answer Relevance
Option B: Agent
- Your app uses tools to accomplish tasks
- It may involve multi-step reasoning or planning
- Examples: research agents, coding assistants, task automation
→ Recommended: Agent GPA metrics (continue to Question 2)
Question 2 (Agents only): Does your agent do explicit planning?
Yes, my agent creates plans before executing:
- Agent outputs a plan/strategy before taking actions
- Agent references its plan during execution
→ Use all Agent GPA metrics:
- Logical Consistency
- Plan Quality
- Plan Adherence
- Execution Efficiency
- Tool Selection
- Tool Calling
- Tool Quality
No, my agent acts without explicit planning:
- Agent takes actions directly without stating a plan
- Agent uses reactive decision-making
→ Use Agent GPA metrics (excluding plan metrics):
- Logical Consistency
- Execution Efficiency
- Tool Selection
- Tool Calling
- Tool Quality
Question 3: Do you want to add any additional evaluations?
Consider adding these based on your needs:
| Evaluation | Use Case |
|---|---|
| Coherence | Check if output is well-structured and readable |
| Conciseness | Ensure responses aren't unnecessarily verbose |
| Harmlessness | Detect potentially harmful content |
| Sentiment | Analyze emotional tone of responses |
| Custom metrics | Domain-specific evaluations (see below) |
Creating Custom Metrics
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 8d ago First seen · 467 lines · 17 tokens per session scan A 5e40ed1636f4
trulens-evaluation-setup is a skill published in the GitHub repository truera/trulens (3,536 stars, last pushed 4d ago), licensed MIT. It adds 17 tokens to every session and 3,080 once invoked, about $0.0001 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
annotate-spans
Write effective, consistent annotations on LLM/agent spans and traces, and coach the user on annotation practice. Load this whenever you are about to record structured feedback with the ui.spans.annotate operation (via executebrowseraction), or when the user asks how to annotate, label, score, or review spans/traces…
playground
Author, edit, or iterate on prompts in the Phoenix prompt playground, including running experiments over a dataset. Load before any playground ui. operation call, including single-shot prompt rewrites.
datasets
Understand what a Phoenix dataset is and reason well about its examples, outputs, splits, and how it feeds evaluators and experiments. Load this whenever a dataset is in view or the user asks what a dataset is, how splits work, what an output "means", or how datasets relate to experiments and evals. This skill governs…
evaluators
Author or refine a Phoenix evaluator — code or LLM-as-a-judge — that scores a run's output. Trigger when the user wants to create a new evaluator, improve an existing one's logic or rubric, choose labels, or decide what to measure on a dataset or experiment. Do NOT trigger on: (1) manual prompt drafting (use…
experiments
Run, read, and compare dataset-backed experiments to find evidence that a prompt or pipeline is improving. Trigger when the user wants to iterate over a dataset with experiments, compare experiment runs, read experiment quality/latency/cost, or decide whether a change actually helped. Running a prompt over a dataset…
ml-expert
Expert-level machine learning, deep learning, model training, and MLOps. Use when the user mentions machine learning, deep learning, neural networks, MLOps, or data science, or when the task involves Machine Learning Fundamentals, Data Preparation, or Model Training.