DeepEval is an open-source framework for testing and measuring large-language-model applications, much like a unit-testing tool specialized for LLM systems. Developers use it to evaluate agents, retrieval-augmented generation pipelines, chatbots, and individual model or tool interactions with metrics such as answer relevance and hallucination.
Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add confident-ai/deepeval --skill deepeval-otelgit clone --depth 1 https://github.com/confident-ai/deepevalWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/confident-ai/deepeval/deepeval-otel)<a href="https://agentmods.dev/skills/confident-ai/deepeval/deepeval-otel"><img src="https://agentmods.dev/badge/skills/confident-ai/deepeval/deepeval-otel/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/confident-ai/deepeval/deepeval-otel"><img src="https://agentmods.dev/badge/skills/confident-ai/deepeval/deepeval-otel.svg" alt="Reviewed on agentmods" width="80" height="20"></a>- NVIDIA SkillSpector pass
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00226 | $0.01689 |
| Opus 5 | $0.00113 | $0.00844 |
| Sonnet 5 | $0.00045 | $0.00338 |
| Haiku 4.5 | $0.00023 | $0.00169 |
Grade A, and why
deepeval-otel scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 11d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 130 lines — stays where its author put it; the contents beside it link to each section on GitHub.
DeepEval OpenTelemetry Export
Use this skill to instrument an AI application — an LLM app, agent, RAG
pipeline, or chatbot — with raw OpenTelemetry so its traces land in
Confident AI's Observatory. No deepeval package is needed — it works with
any OTLP-capable OpenTelemetry SDK. The job is exactly two things: export to
the correct Confident AI OTLP endpoint, and set the confident.* attributes
Confident AI reads off each span.
Scope: AI Applications Only
This skill instruments AI applications only. The confident.* attributes
and span types — agent, llm, retriever, tool — describe AI components,
and Confident AI's Observatory is built to evaluate and monitor AI behavior.
Instrument only the AI parts of the system: agent loops and planning, LLM
calls, retrieval / vector search, and tool calls. Do not apply confident.*
attributes to non-AI software (web servers, CRUD backends, database layers,
infrastructure) or to non-AI spans inside an otherwise-AI app — that data does
not belong in Confident AI and will not render meaningfully. If the target has
no LLM, agent, retrieval, or tool-calling component, this skill does not apply.
When to Use vs the deepeval Skill
Use this skill for vendor-neutral OTLP export to Confident AI — pointing an
OpenTelemetry exporter at Confident AI and setting confident.* attributes.
Use the deepeval skill when the user wants to build a Python pytest eval
suite, generate datasets or goldens, write metrics, run deepeval test run, or
instrument with the deepeval SDK's @observe decorator. The two skills are
complementary, not alternatives.
Prerequisites
- A Confident AI account and a
CONFIDENT_API_KEY. - An OpenTelemetry SDK for the application's language. For Python:
opentelemetry-sdkandopentelemetry-exporter-otlp-proto-http. - The Confident AI OTLP endpoint accepts HTTP only — never gRPC.
How It Works
Confident AI exposes an OTLP/HTTP traces endpoint. Point any OpenTelemetry span
exporter at it with the x-confident-api-key header. Confident AI's exporter
then reads confident.* attributes off each span to build the trace and span
structure. Parent/child nesting comes from native OpenTelemetry span context,
not from any attribute.
What ships with it
6 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 11d ago First seen · 130 lines · 226 tokens per session scan A 9e9520207797
deepeval-otel is a skill published in the GitHub repository confident-ai/deepeval (18,203 stars, last pushed 2d ago), licensed Apache-2.0. It adds 226 tokens to every session and 1,689 once invoked, about $0.0011 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
cuml-machine-learning
Use for GPU-accelerated machine learning on tabular data using NVIDIA cuML. Triggers when tasks involve classification, regression, clustering, dimensionality reduction, or model training on datasets.
optimize-for-gpu
GPU-accelerates scientific Python on NVIDIA hardware and verifies that the result is correct and faster. Use for CUDA/GPU optimization; CPU-bound NumPy, SciPy, pandas, scikit-learn, NetworkX, scikit-image, vector-search, image-processing, graph, simulation, or file-I/O workloads; CuPy, cuDF, cuML, cuGraph, cuVS…
thinking-out-loud
A contract for what the agent does when a long, messy, stream-of-consciousness ramble arrives (usually voice dictation): act on nothing until the echo brief is approved. The echo audits the entire transfer, mission, locked decisions and constraints, open questions, flips and parked tangents, with the model's…
pydantic-ai
Build production-ready AI agents with PydanticAI — type-safe tool use, structured outputs, dependency injection, and multi-model support.
open-source
Documentation reference for writing Python code using the browser-use open-source library. Use this skill whenever the user needs help with Agent, Browser, or Tools configuration, is writing code that imports from browseruse, asks about @sandbox deployment, supported LLM models, Actor API, custom tools, lifecycle…
marimo-pair
Work inside the user's live marimo notebook from the code editor: run Python in the same kernel the user does, inspect live notebook state, and commit durable notebook changes through code mode. Use whenever you create, analyze, or improve the user's marimo notebook.