Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add krzysztofsurdy/code-virtuoso --skill llm-observabilitygit clone --depth 1 https://github.com/krzysztofsurdy/code-virtuosoWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/krzysztofsurdy/code-virtuoso/llm-observability)<a href="https://agentmods.dev/skills/krzysztofsurdy/code-virtuoso/llm-observability"><img src="https://agentmods.dev/badge/skills/krzysztofsurdy/code-virtuoso/llm-observability/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/krzysztofsurdy/code-virtuoso/llm-observability"><img src="https://agentmods.dev/badge/skills/krzysztofsurdy/code-virtuoso/llm-observability.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00049 | $0.02334 |
| Opus 5.5 | $0.00020 | $0.00934 |
| Sonnet 5.5 | $0.00010 | $0.00467 |
| Haiku 4.5 | $0.00005 | $0.00233 |
Grade A, and why
llm-observability scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured yesterday.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 152 lines — stays where its author put it; the contents beside it link to each section on GitHub.
LLM observability
An LLM app fails in ways that logs and error rates do not show. The request returns 200, the answer is wrong, the agent looped nine times, and the bill doubled. To see these you need a trace of every model call and tool call, token counts on each, and a way to connect the trace to quality checks. This skill covers how to record that data, what to measure, and what to keep out of it.
Core Principles
| Principle | Meaning |
|---|---|
| Trace the whole run | One trace per user request or agent run, with a span per model call and per tool call. A flat log of prompts cannot show where a run went wrong. |
| Concepts over attribute names | The OpenTelemetry GenAI conventions are still in Development status and names change. Learn the shape, then take exact names from the current spec. |
| Tokens are the source of truth for cost | Record input, output, cached, and reasoning tokens on every call. Compute money later from a versioned price table. |
| Measure before optimizing | Cache hit ratio, time to first token, and step count are numbers you read from traces. Do not tune on a guess. |
| Content is opt-in | Prompts and completions hold user data. Capture them only on purpose, with redaction, retention, and access rules. |
| Keep the interesting traces | Sample so that errors, slow runs, costly runs, and bad-feedback runs are always kept. |
| Close the loop | Production traces feed error analysis and the eval datasets. See llm-evals. |
What a trace looks like
invoke_agent support-bot (one user request)
chat model-x (model call 1: plans, picks a tool)
execute_tool search_orders (tool call)
chat model-x (model call 2: reads result)
execute_tool issue_refund (tool call)
chat model-x (model call 3: final answer)
Span types to emit:
| Span | Covers | Key data |
|---|---|---|
| Workflow or agent run | One request through one agent or pipeline | Agent name, conversation id, step count, stop reason, total tokens |
| Model call | One request to a model | Provider, requested and actual model, token counts (input, output, cached read and write, reasoning), finish reason, parameters, error type |
| Tool call | One tool execution | Tool name, call id, duration, error. Arguments and results are content, so opt-in |
| Retrieval, embedding, guardrail | Supporting steps | Duration, counts, index or model used |
What ships with it
4 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- yesterday First seen · 152 lines · 49 tokens per session scan A c58b7093ceaa
llm-observability is a skill published in the GitHub repository krzysztofsurdy/code-virtuoso (36 stars, last pushed today), licensed MIT. It adds 49 tokens to every session and 2,334 once invoked, about $0.0002 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-10-22.
Other skills, from other repositories
gh-issues
Fetch GitHub issues, spawn sub-agents to implement fixes and open PRs, then monitor and address PR review comments. Usage: /gh-issues [owner/repo] [--label bug] [--limit 5] [--milestone v1.0] [--assignee @me] [--fork user/repo] [--watch] [--interval 5] [--reviews-only] [--cron] [--dry-run] [--model glm-5]…
oracle
Best practices for using the oracle CLI (prompt + file bundling, engines, sessions, and file attachment patterns).
model-usage
Use CodexBar CLI local cost usage to summarize per-model usage for Codex or Claude, including the current (most recent) model or a full model breakdown. Trigger when asked for model-level usage/cost data from codexbar, or when you need a scriptable per-model summary from codexbar cost JSON.
hf-model-card-research
Extract structured metadata — downloads, likes, benchmark claims, file sizes, author statements — from HuggingFace model cards. Used when the user asks you to "check these models on HF", "pull benchmarks for these variants", or "compare what authors claim.".
mnemosyne-maintenance
Use when: upgrading Mnemosyne, diagnosing slow/hung consolidation (mnemosynesleep), fixing missing embeddings, or troubleshooting import/version mismatches.
agent-platform-tuning
Agent Platform Model Tuning. Use when you need to fine-tune open models or Gemini models using Agent Platform infrastructure. Don't use for model training outside Agent Platform, model deployment to endpoints (use agent-platform-deploy), or managing serving endpoints (use agent-platform-endpoint-management).