redis-semantic-cache

Redis Semantic Cache

Semantic caching for LLM responses with Redis Cloud's LangCache service. Stores prompts as embeddings; subsequent semantically-similar prompts return the cached response without re-calling the model.

LangCache is currently in preview on Redis Cloud. Features and behavior may change.

When to apply

• Wrapping an LLM call (OpenAI, Anthropic, etc.) with a cache layer to cut cost and latency.
• Caching RAG answers, classification outputs, or any deterministic LLM workload.
• Tuning the precision/hit-rate trade-off for a semantic cache.
• Splitting one application's LLM workloads across multiple cache instances.

1. The cache-aside flow

LangCache fits in front of any LLM call as a standard cache-aside pattern:

1. Send the user's prompt to LangCache's search.
2. Cache hit — return the stored response directly.
3. Cache miss — call the LLM, then set the response so future similar prompts hit.

from langcache import LangCache
import os

lang_cache = LangCache(
    server_url=f"https://{os.getenv('HOST')}",
    cache_id=os.getenv("CACHE_ID"),
    api_key=os.getenv("API_KEY"),
)

result = lang_cache.search(prompt="What is Redis?", similarity_threshold=0.9)
if result:
    response = result[0]["response"]
else:
    response = llm.generate("What is Redis?")
    lang_cache.set(prompt="What is Redis?", response=response)

The same operations are available via REST (POST /v1/caches/{cacheId}/entries/search and POST /v1/caches/{cacheId}/entries) when an SDK isn't an option.

See references/langcache-usage.md for full SDK + REST samples and attribute-based storage.

2. Tune the similarity threshold

The threshold controls how close (in embedding cosine distance) a new prompt must be to a cached one to count as a hit. Higher = stricter match, fewer false positives. Lower = more hits, more risk of returning an off-topic answer.

Threshold	Behavior	Use when
0.95+	Near-exact match required	Customer-facing answers where wrong responses are costly
0.9	Balanced default	Most workloads — start here
0.8	Loose semantic match	Internal tools, exploratory queries, FAQ deduplication