TruLens is an open-source system for tracing and evaluating LLM applications and AI agents. It records each step's inputs, outputs, latency, tokens, and cost, then uses evaluations to find failures and compare application versions.
Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add truera/trulens --skill relevance-feedbackgit clone --depth 1 https://github.com/truera/trulensWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/truera/trulens/relevance-feedback)<a href="https://agentmods.dev/skills/truera/trulens/relevance-feedback"><img src="https://agentmods.dev/badge/skills/truera/trulens/relevance-feedback/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/truera/trulens/relevance-feedback"><img src="https://agentmods.dev/badge/skills/truera/trulens/relevance-feedback.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00146 | $0.02212 |
| Opus 5.5 | $0.00058 | $0.00885 |
| Sonnet 5.5 | $0.00029 | $0.00442 |
| Haiku 4.5 | $0.00015 | $0.00221 |
Grade A, and why
qdrant-relevance-feedback scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 22d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
Copies of this mod
2 near-identical copies found in the catalogue:
- qdrant-relevance-feedback — 100% identical, 10 lines differ
- qdrant-relevance-feedback — 100% identical, 70 lines differ
How it starts
The opening of the file, as written. The whole thing — 119 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Reranking reorders documents that have already been retrieved. Qdrant's Relevance Feedback (RF) instead modifies the vector search process itself based on a small amount of reranker feedback, distilling reranker (feedback model) knowledge into the search step. This allows RF to surface documents that the initial ANN search did not score highly enough.
The RF is intended for tasks where relevance correlates with similarity in vector space.
How you apply the RF depends on your goals. First, understand how the RF works, read the ENTIRE section. Then define your goals and choose the appropriate usage pattern described below. Make sure to avoid the listed anti-patterns ("DO NOTs"). Before implementing anything, read CAREFULLY to avoid missing important details.
How It Works
The Qdrant Query Point API with a type RelevanceFeedbackQuery takes:
- a query (
target) - a small list of seed documents (
feedback) with relevance scores (often 4–5 seeds are enough) - formula weights, which MUST be trained once per general search use case (your dataset, dense retriever, and feedback model)
If you do not train the formula weights, results will at best be random, will not align with your data distribution or model behavior. Training is lightweight because the formula itself is simple.
During search, it scores each candidate by combining similarity to the original query, similarity to highly rated seed documents and dissimilarity to poorly rated ones.
Feedback Model
A feedback model is any model that can produce a float relevance score for (query, document) pairs. Higher scores must always mean higher relevance.
Examples: a cross-encoder, embedding similarity (for example, cosine similarity between query and document embeddings, or max_sim for late interaction models), an LLM-based scorer, a custom ranker.
The feedback model used during training and inference MUST be the same model. Formula weights during training are calibrated to that model's score distribution. If you switch feedback models, you must retrain.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 22d ago First seen · 119 lines · 146 tokens per session scan A 34e31c62d181
qdrant-relevance-feedback is a skill published in the GitHub repository truera/trulens (3,602 stars, last pushed today), licensed MIT. It adds 146 tokens to every session and 2,212 once invoked, about $0.0006 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-24.
Other skills, from other repositories
playground
Author, edit, or iterate on prompts in the Phoenix prompt playground, including running experiments over a dataset. Load before any playground ui. operation call, including single-shot prompt rewrites.
datasets
Understand what a Phoenix dataset is and reason well about its examples, outputs, splits, and how it feeds evaluators and experiments. Load this whenever a dataset is in view or the user asks what a dataset is, how splits work, what an output "means", or how datasets relate to experiments and evals. This skill governs…
evaluators
Author or refine a Phoenix evaluator — code or LLM-as-a-judge — that scores a run's output. Trigger when the user wants to create a new evaluator, improve an existing one's logic or rubric, choose labels, or decide what to measure on a dataset or experiment. Do NOT trigger on: (1) manual prompt drafting (use…
experiments
Run, read, and compare dataset-backed experiments to find evidence that a prompt or pipeline is improving. Trigger when the user wants to iterate over a dataset with experiments, compare experiment runs, read experiment quality/latency/cost, or decide whether a change actually helped. Running a prompt over a dataset…
flow-nexus-neural
Train and deploy neural networks in distributed E2B sandboxes with Flow Nexus.
hugging-face-model-trainer
This skill should be used when users want to train or fine-tune language models using TRL (Transformer Reinforcement Learning) on Hugging Face Jobs infrastructure. Covers SFT, DPO, GRPO and reward modeling training methods, plus GGUF conversion for local deployment. Includes guidance on the TRL Jobs package, UV…