Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add AnthonyAlcaraz/agentic-graph-rag-skills --skill kv-cache-latency-budgetergit clone --depth 1 https://github.com/AnthonyAlcaraz/agentic-graph-rag-skillsWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/anthonyalcaraz/agentic-graph-rag-skills/kv-cache-latency-budgeter)<a href="https://agentmods.dev/skills/anthonyalcaraz/agentic-graph-rag-skills/kv-cache-latency-budgeter"><img src="https://agentmods.dev/badge/skills/anthonyalcaraz/agentic-graph-rag-skills/kv-cache-latency-budgeter/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/anthonyalcaraz/agentic-graph-rag-skills/kv-cache-latency-budgeter"><img src="https://agentmods.dev/badge/skills/anthonyalcaraz/agentic-graph-rag-skills/kv-cache-latency-budgeter.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00173 | $0.02344 |
| Opus 5 | $0.00086 | $0.01172 |
| Sonnet 5 | $0.00035 | $0.00469 |
| Haiku 4.5 | $0.00017 | $0.00234 |
Grade A, and why
kv-cache-latency-budgeter scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 12d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 170 lines — stays where its author put it; the contents beside it link to each section on GitHub.
KV-Cache Latency Budgeter
Overview
The architecture distributes work across two channels that both bottleneck at production scale: graph traversals that navigate the knowledge graph, and model-inference calls that power the workflow nodes.
Graph analytics (PageRank, community detection, centrality) are bound by memory bandwidth, not compute — irregular random access CPUs handle poorly and GPUs handle well. cuGraph / nx-cugraph report order-of-magnitude speedups: PageRank 137x on A100, Louvain 125x over NetworkX, multi-GPU PageRank 80x over a 100-node Spark cluster, and betweenness centrality on the LiveJournal graph (4.8M nodes, 69M edges) from 7 minutes to 5 seconds — a 485x speedup. For the DevOps agent, blast-radius analysis drops from 3-5 seconds on CPU to under 100 ms on one GPU.
Inference latency is bound by the KV cache. Multi-LoRA serving lowers cost per weight but not the binding constraint: peak KV per active user, not model size, sets how many concurrent analyses one H100 (80 GB) can host. Quantizing the weights does not move this ceiling — you have to bound the cache itself. The chapter's first production-ready recipe is Microsoft MEMENTO: a two-stage supervised fine-tune on 228,000 traces teaches the model to segment its chain of thought into blocks, emit a compressed summary token per block, and mask the original block from future attention, producing a sawtooth KV pattern.
The chapter's source redacts the exact MEMENTO reduction factor, per-block compression factor, and concurrent-incident multiplier. This skill treats those as caller-measured inputs and never fabricates them — consistent with the chapter's own [Tip]: measure peak KV per active user first.
When to Use
- Scaling a multi-model agent and deciding how many concurrent incidents one GPU can host.
- Choosing between weight quantization and KV compression to raise concurrency.
- Estimating whether GPU-accelerated graph analytics fit the latency budget.
- Setting an end-to-end latency target for a multi-step workflow.
What ships with it
2 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 12d ago First seen · 170 lines · 173 tokens per session scan A 3152e2849a50
kv-cache-latency-budgeter is a skill published in the GitHub repository AnthonyAlcaraz/agentic-graph-rag-skills (10 stars, last pushed 2mo ago), licensed MIT. It adds 173 tokens to every session and 2,344 once invoked, about $0.0009 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
Other skills, from other repositories
cost-optimization
Optimize cloud costs across AWS, Azure, GCP, and OCI through resource rightsizing, tagging strategies, reserved instances, and spending analysis. Use when reducing cloud expenses, analyzing infrastructure costs, or implementing cost governance policies.
istio-traffic-management
Configure Istio traffic management including routing, load balancing, circuit breakers, and canary deployments. Use when implementing service mesh traffic policies, progressive delivery, or resilience patterns.
linkerd-patterns
Implement Linkerd service mesh patterns for lightweight, security-focused service mesh deployments. Use when setting up Linkerd, configuring traffic policies, or implementing zero-trust networking with minimal overhead.
nemo-automodel-launcher-config
Configure NeMo AutoModel job launches for interactive runs, Slurm clusters, and SkyPilot cloud execution.
artifact-deploy
One-click deploy a user's pre-built app/artifact into their OWN AWS account and get a global public HTTPS link (Vercel-like), with a default TTL and promote-to-persistent. Use when the user says "deploy this", "ship this demo", "give me a public link", "share this externally", or "deploy to AWS".
tcapi
A helper for managing Tencent Cloud resources through Tencent Cloud's command-line API tool. Tencent Cloud is a cloud-services provider offering products such as virtual servers, storage, networks, and containers.