Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add nexus-labs-automation/agent-observability --skill memory-rag-instrumentationgit clone --depth 1 https://github.com/nexus-labs-automation/agent-observabilityWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/nexus-labs-automation/agent-observability/memory-rag-instrumentation)<a href="https://agentmods.dev/skills/nexus-labs-automation/agent-observability/memory-rag-instrumentation"><img src="https://agentmods.dev/badge/skills/nexus-labs-automation/agent-observability/memory-rag-instrumentation/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/nexus-labs-automation/agent-observability/memory-rag-instrumentation"><img src="https://agentmods.dev/badge/skills/nexus-labs-automation/agent-observability/memory-rag-instrumentation.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00018 | $0.01477 |
| Opus 5 | $0.00009 | $0.00739 |
| Sonnet 5 | $0.00004 | $0.00295 |
| Haiku 4.5 | $0.00002 | $0.00148 |
Grade A, and why
memory-rag-instrumentation scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 12d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 202 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Memory and RAG Instrumentation
Instrument retrieval-augmented generation and memory operations for quality debugging.
Core Principle
RAG observability answers:
- What was retrieved? (sources, scores)
- Was it relevant? (quality signals)
- How much context was used?
- Did retrieval affect the response?
Retrieval Span Attributes
# Required (P0)
span.set_attribute("retrieval.source", "vector_store")
span.set_attribute("retrieval.query_length", 150)
span.set_attribute("retrieval.results_count", 5)
span.set_attribute("retrieval.latency_ms", 45)
# Quality signals (P1)
span.set_attribute("retrieval.top_score", 0.89)
span.set_attribute("retrieval.avg_score", 0.72)
span.set_attribute("retrieval.min_score", 0.55)
span.set_attribute("retrieval.above_threshold", 4) # Count above relevance threshold
# Context usage (P1)
span.set_attribute("retrieval.tokens_retrieved", 2500)
span.set_attribute("retrieval.tokens_used", 2000) # After truncation
span.set_attribute("retrieval.context_window_pct", 0.25) # % of context window
# Source tracking (P2)
span.set_attribute("retrieval.sources", ["doc1.pdf", "doc2.pdf"])
span.set_attribute("retrieval.collection", "knowledge_base")
Retrieval Pipeline Stages
Query Processing
with tracer.start_span("retrieval.query_process") as span:
span.set_attribute("query.original_length", len(query))
span.set_attribute("query.expanded", bool(expansion))
span.set_attribute("query.rewritten", bool(rewrite))
# Process query
Vector Search
with tracer.start_span("retrieval.vector_search") as span:
span.set_attribute("vector.index", "main_index")
span.set_attribute("vector.k", 10)
span.set_attribute("vector.ef_search", 100) # HNSW param
span.set_attribute("vector.distance_metric", "cosine")
# Execute search
Reranking
with tracer.start_span("retrieval.rerank") as span:
span.set_attribute("rerank.model", "cohere-rerank-v3")
span.set_attribute("rerank.input_count", 10)
span.set_attribute("rerank.output_count", 5)
span.set_attribute("rerank.score_improvement", 0.15)
# Rerank results
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 12d ago First seen · 202 lines · 18 tokens per session scan A 0e3849c15822
memory-rag-instrumentation is a skill published in the GitHub repository nexus-labs-automation/agent-observability (7 stars, last pushed 8mo ago), licensed MIT. It adds 18 tokens to every session and 1,477 once invoked, about $0.0001 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
Other skills, from other repositories
AgentDB Memory Patterns
Implement persistent memory patterns for AI agents using AgentDB. Includes session memory, long-term storage, pattern learning, and context management. Use when building stateful agents, chat systems, or intelligent assistants.
pinecone-research
Agent RAG and long-term memory with Pinecone.
agent-v3-memory-specialist
Agent skill for v3-memory-specialist - invoke with $agent-v3-memory-specialist.
langchain
Framework for building LLM-powered applications with agents, chains, and RAG. Supports multiple providers (OpenAI, Anthropic, Google), 500+ integrations, ReAct agents, tool calling, memory management, and vector store retrieval. Use for building chatbots, question-answering systems, autonomous agents, or RAG…
llm-wiki
The foundational knowledge distillation pattern for building and maintaining an AI-powered Obsidian wiki. Based on Andrej Karpathy's LLM Wiki architecture. Use this skill whenever the user wants to understand the wiki pattern, set up a new knowledge base, or needs guidance on the three-layer architecture (raw sources…
browserwing-admin
Manage and operate BrowserWing — an intelligent browser automation platform. Install dependencies, configure LLM, create/manage/execute automation scripts, use AI-driven exploration to generate scripts, browse the script marketplace, and troubleshoot issues.