Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add instructions/cloud-apim/otoroshi-llm-extension/agents-mdgit clone --depth 1 https://github.com/cloud-apim/otoroshi-llm-extensionWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/instructions/cloud-apim/otoroshi-llm-extension/agents-md)<a href="https://agentmods.dev/instructions/cloud-apim/otoroshi-llm-extension/agents-md"><img src="https://agentmods.dev/badge/instructions/cloud-apim/otoroshi-llm-extension/agents-md.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5 | $0.00934 | $0.00934 |
| Opus 5 | $0.00467 | $0.00467 |
| Sonnet 5 | $0.00187 | $0.00187 |
| Haiku 4.5 | $0.00093 | $0.00093 |
Grade A, and why
otoroshi-llm-extension AGENTS.md scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 4d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 87 lines — stays where its author put it; the contents beside it link to each section on GitHub.
AGENTS.md
This file provides guidance to Codex (Codex.ai/code) when working with code in this repository.
Project Overview
Otoroshi LLM Extension is a set of Otoroshi plugins for building an AI/LLM gateway. It provides a unified interface to 20+ LLM providers with features like load balancing, fallbacks, semantic caching, guardrails, and budget management.
Requirements: JDK 17+ and Scala 3.8.4
Build Commands
# Compile the project
sbt compile
# Run tests (tests run sequentially, not in parallel)
sbt test
# Run a single test suite
sbt "testOnly com.cloud.apim.otoroshi.extensions.aigateway.ProvidersSuite"
# Run only the suites that do not need a real LLM provider
sh ./scripts/run-offline-tests.sh
# Build fat JAR for deployment
sbt assembly
# Output: target/scala-3.8.4/otoroshi-llm-extension-assembly_3-dev.jar
Architecture
Package Structure
All source code lives under src/main/scala/com/cloud/apim/otoroshi/extensions/aigateway/:
- extension.scala - Main entry point (
AiExtension), extends Otoroshi'sAdminExtension - models.scala - Core chat abstractions (
ChatMessage,ChatPrompt,ChatResponse,ChatClient) - providers/ - LLM provider implementations (OpenAI, Anthropic, Azure, Mistral, Groq, etc.)
- plugins/ - Otoroshi plugins (
LLMProxy,OpenAiCompatibleProxy, rate limiting, audio, images, etc.) - decorators/ - Cross-cutting concerns using decorator pattern (caching, guardrails, memory, costs)
- guardrails/ - Validation implementations (LLM-based, regex, moderation, webhook, WASM)
- entities/ - Configuration entities stored in Redis (providers, prompts, templates, budgets)
- agents/ - Agent implementations for agentic workflows
- workflows.scala - Otoroshi workflow function integrations
Key Patterns
Decorator Chain (decorators/decorators.scala): Chat clients are wrapped with decorators for cross-cutting concerns. The chain is applied in ChatClientDecorators.withDecorators():
- Model Constraints → 2. Provider Fallback → 3. Persistent Memory → 4. Semantic Cache → 5. Simple Cache → 6. Guardrails → 7. Costs → 8. Auditing
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 4d ago First seen · 87 lines · 934 tokens per session scan A c80a2cc1823d
otoroshi-llm-extension AGENTS.md is an instructions file published in the GitHub repository cloud-apim/otoroshi-llm-extension (18 stars, last pushed 14d ago), licensed Apache-2.0. It adds 934 tokens to every session, about $0.0047 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other instructions, from other repositories
higress AGENTS.md
AGENTS.md instructions for higress-group/higress, covering agents.md, mandatory issue-spec gate for agent-assisted changes, repository layout, plugins and plugins/wasm-go/ (primary wasm plugin framework, go).
ai-gateway AGENTS.md
AGENTS.md instructions for ferro-labs/ai-gateway, covering agents.md, project overview, current development snapshot, public-facing wording and build, test, and run commands.
litellm AGENTS.md
AGENTS.md instructions for BerriAI/litellm, a project described as: The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM].
litellm GEMINI.md
Gemini CLI instructions for BerriAI/litellm, a project described as: The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM].
awesome-ai-gateway CLAUDE.md
Claude Code instructions for cuihuan/awesome-ai-gateway, covering claude.md — working rules for this repository, what this repository is, the two goals — judge every change against these, non-negotiable rules and before you commit.
sbproxy AGENTS.md
Instructions for soapbucket/sbproxy, covering sbproxy (rust workspace), pre-commit checks, changelog entries are fragments, not changelog.md edits, the gate validates the working tree; git push ships head and running the gate.