Borrowing it
Nothing to install: this file belongs to majiayu000/litellm-rs. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.
curl -O https://raw.githubusercontent.com/majiayu000/litellm-rs/main/.claude/skills/caching-architecture/SKILL.mdgit clone --depth 1 https://github.com/majiayu000/litellm-rsWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/majiayu000/litellm-rs/caching-architecture)<a href="https://agentmods.dev/skills/majiayu000/litellm-rs/caching-architecture"><img src="https://agentmods.dev/badge/skills/majiayu000/litellm-rs/caching-architecture/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/majiayu000/litellm-rs/caching-architecture"><img src="https://agentmods.dev/badge/skills/majiayu000/litellm-rs/caching-architecture.svg" alt="Reviewed on agentmods" width="80" height="20"></a>- NVIDIA SkillSpector pass
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00098 | $0.01960 |
| Opus 5 | $0.00049 | $0.00980 |
| Sonnet 5 | $0.00020 | $0.00392 |
| Haiku 4.5 | $0.00010 | $0.00196 |
Grade A, and why
caching-architecture scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 11d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 134 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Caching Architecture Guide
Overview
LiteLLM-RS ships exactly one wired caching subsystem: an exact-match response cache for non-streaming chat completions and embeddings. It is a two-tier read-through cache, not a three-tier stack — semantic (vector) caching exists as a deprecated, unwired module (see below).
Request (non-streaming /v1/chat/completions, /v1/embeddings)
│ lookup_chat / lookup_embedding (src/server/routes/ai/response_cache.rs)
▼
┌────────────────────────────────────────────────────────────┐
│ LLMCache (src/core/cache/llm_cache.rs) │
│ chat_cache: DualCache<CachedChatResponse> │
│ embedding_cache: DualCache<CachedEmbeddingResponse> │
└────────────────────────────────────────────────────────────┘
│ per-key get / set
▼
┌────────────────────────────────────────────────────────────┐
│ DualCache<T> (src/core/cache/dual.rs) │
│ L1 InMemoryCache<T> — DashMap, TTL, sampled eviction │
│ L2 RedisCache<T> — optional, backed by RedisPool │
│ Read: L1 miss → L2 hit → repopulate L1 │
│ Write: both tiers; L2 failure logs a warning, not fatal │
└────────────────────────────────────────────────────────────┘
│ miss
▼
LLM Provider → response stored back into both tiers
What Is Wired vs Not
| Capability | Status |
|---|---|
| Exact-match response cache (chat + embeddings) | Wired: AppState.response_cache, built by build_response_cache (src/server/state.rs:143) |
| Semantic similarity cache | Not wired: cache.semantic_cache: true fails startup validation (src/config/validation/cache_validators.rs:16); core::semantic_cache is deprecated since 0.6.0, removal planned in 0.7.0 (src/core/semantic_cache/mod.rs) |
| Vector DB backends | Storage-only: QdrantStore implemented; weaviate/pinecone declared but return "not implemented yet" (src/storage/vector/backend.rs:29). Nothing connects them to caching at runtime |
| Cloud object-storage caches | core::cache::cloud (CloudCache trait; S3/GCS/Azure under feature s3) — not part of the request path |
What ships with it
6 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 11d ago First seen · 134 lines · 98 tokens per session scan A c59cdb78fcf9
caching-architecture is a skill published in the GitHub repository majiayu000/litellm-rs (112 stars, last pushed 2d ago), licensed MIT. It adds 98 tokens to every session and 1,960 once invoked, about $0.0005 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
spring-cache
Spring Cache abstraction for Spring Boot 3.x. Covers @Cacheable, @CacheEvict, @CachePut, cache managers (Caffeine, Redis, EhCache), TTL configuration, cache keys, conditional caching, and cache synchronization. USE WHEN: user mentions "spring cache", "@Cacheable", "@CacheEvict", "cache manager", "Caffeine cache"…
caching-strategies
Application caching patterns. Redis caching, in-memory caches, HTTP caching, cache invalidation strategies, cache-aside, write-through, and CDN caching. USE WHEN: user mentions "caching", "cache invalidation", "Redis cache", "HTTP cache", "CDN caching", "cache-aside", "write-through", "TTL", "stale-while-revalidate"…
caching-strategy
Stratégie de cache adaptée à chaque cas d'usage (Redis, Memcached, in-memory, CDN) — quoi cacher, durée de vie, invalidation et cohérence. Se déclenche avec "cache", "Redis", "caching", "mise en cache", "cache invalidation", "CDN", "distributed cache", "réduire les appels à la base". Also triggers on "caching…
redis-patterns
Patterns d'utilisation Redis pour le cache, pub/sub, streams et sessions. Se déclenche avec "Redis", "cache distribué", "pub/sub Redis", "Redis streams", "session store. Also triggers on "Redis cache", "Redis pub/sub".
redis-patterns
Redis patterns — caching (cache-aside, write-through), sessions, pub/sub, work queues, distributed locks, data structures (sorted sets, streams, hashes). Covers ioredis, node-redis, Redis 7+. Use when designing Redis usage in Node.js or Python apps.
caching-strategies
Implement effective caching at every layer: in-memory, Redis, CDN, and browser. Activate whenever the user asks about caching, performance optimization for repeated data access, Redis patterns, CDN configuration, cache invalidation, HTTP cache headers, memoization, or stale-while-revalidate strategies.