DeepEval is an open-source framework for testing and measuring large-language-model applications, much like a unit-testing tool specialized for LLM systems. Developers use it to evaluate agents, retrieval-augmented generation pipelines, chatbots, and individual model or tool interactions with metrics such as answer relevance and hallucination.
Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add confident-ai/deepeval --skill deepeval-tracinggit clone --depth 1 https://github.com/confident-ai/deepevalWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/confident-ai/deepeval/deepeval-tracing)<a href="https://agentmods.dev/skills/confident-ai/deepeval/deepeval-tracing"><img src="https://agentmods.dev/badge/skills/confident-ai/deepeval/deepeval-tracing/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/confident-ai/deepeval/deepeval-tracing"><img src="https://agentmods.dev/badge/skills/confident-ai/deepeval/deepeval-tracing.svg" alt="Reviewed on agentmods" width="80" height="20"></a>- NVIDIA SkillSpector pass
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00208 | $0.01241 |
| Opus 5 | $0.00104 | $0.00620 |
| Sonnet 5 | $0.00042 | $0.00248 |
| Haiku 4.5 | $0.00021 | $0.00124 |
Grade A, and why
deepeval-tracing scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 10d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 100 lines — stays where its author put it; the contents beside it link to each section on GitHub.
DeepEval Tracing
Use this skill to instrument an AI application — an LLM app, agent, RAG
pipeline, or chatbot — with DeepEval's native tracing so its execution is
visible span by span in Confident AI's Observatory. The work is: pick a
supported integration when one exists, fall back to manual @observe
otherwise, give each span a meaningful type, and add tags and metadata.
This skill stops at producing well-formed traces. Attaching evaluation metrics
and running evals is the deepeval skill's job.
Scope: AI Applications Only
Instrument only the AI parts of the system — agent loops and planning, LLM
calls, retrieval / vector search, and tool calls. The span types (llm,
retriever, tool, agent) describe AI components. Do not trace non-AI
software (web servers, CRUD backends, infrastructure). If the target has no
LLM, agent, retrieval, or tool-calling component, this skill does not apply.
When to Use vs the deepeval and deepeval-otel Skills
- This skill (
deepeval-tracing) — instrument an app with the DeepEval SDK (@observe, framework integrations) so traces reach Confident AI. deepevalskill — build pytest eval suites: datasets, metrics, traced evals,deepeval test run, iteration. It runs evals against an app this skill instrumented.deepeval-otelskill — instrument with the vendor-neutral OpenTelemetry SDK instead of the DeepEval SDK (raw OTLP, including non-Python apps).
The three are complementary. If unsure between this skill and deepeval-otel:
use this one when the app is Python and you want the DeepEval SDK; use
deepeval-otel when you want raw OpenTelemetry or the app is not Python.
Prerequisites
- An AI application in Python with
pip install deepeval. - For traces to reach Confident AI:
deepeval login, or an exportedCONFIDENT_API_KEY(preferred for CI and non-interactive runs).
Workflow
- Confirm the target is an AI application (it has LLM calls, an agent loop, retrieval, or tool calls). If it has none of these, stop — this skill does not apply.
- Detect the framework, model provider, agent SDK, and vector database in use.
- Read
references/integrations.mdand the exact integration doc for what was detected. Prefer a native integration over manual instrumentation. - If no native integration fits, instrument manually with
@observe. Readreferences/tracing.md. - Give each span a meaningful
type(llm,retriever,tool,agent) and capture inputs/outputs. - Add trace-level tags and metadata where they help diagnose failure patterns. Never trace secrets, credentials, or raw sensitive data.
- Confirm
deepeval loginorCONFIDENT_API_KEY, then verify traces appear in the Confident AI Observatory.
What ships with it
3 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 10d ago First seen · 100 lines · 208 tokens per session scan A 57ccb0207cc7
deepeval-tracing is a skill published in the GitHub repository confident-ai/deepeval (18,186 stars, last pushed yesterday), licensed Apache-2.0. It adds 208 tokens to every session and 1,241 once invoked, about $0.0010 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
cocoindex
This skill should be used when building data processing pipelines with CocoIndex, a Python library for incremental data transformation. Use when the task involves processing files/data into databases, creating vector embeddings, building knowledge graphs, ETL workflows, or any data pipeline requiring automatic change…
chromadb
Operational skill for ChromaDB: collections, embeddings, metadata filters, persistence, and local RAG vector storage.
sql-reporting
Conventions and review steps for writing analytics SQL against the warehouse. Use whenever the task involves querying tables, building a report, or aggregating metrics.
schema-exploration
Lists tables, describes columns and data types, identifies foreign key relationships, and maps entity relationships in a database. Use when the user asks about database schema, table structure, column types, what tables exist, ERD, foreign keys, or how entities relate.
deepagents-thread-inspector
Inspect and explain conversations in the local Deep Agents Code SQLite session store. Use as a fallback when LangSmith trace tooling is unavailable, for offline or untraced sessions, or when asked to identify or summarize a local dcode thread, inspect checkpoint metadata, list recent local threads, or parse…
query-writing
Writes and executes SQL queries from simple SELECTs to complex multi-table JOINs, aggregations, and subqueries. Use when the user asks to query a database, write SQL, run a SELECT statement, retrieve data, filter records, or generate reports from database tables.