Borrowing it
Nothing to install: this file belongs to TakaGoto/rag-learning-academy. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.
curl -O https://raw.githubusercontent.com/TakaGoto/rag-learning-academy/main/.claude/agents/multimodal-specialist.mdgit clone --depth 1 https://github.com/TakaGoto/rag-learning-academyWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/agents/takagoto/rag-learning-academy/multimodal-specialist)<a href="https://agentmods.dev/agents/takagoto/rag-learning-academy/multimodal-specialist"><img src="https://agentmods.dev/badge/agents/takagoto/rag-learning-academy/multimodal-specialist/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/agents/takagoto/rag-learning-academy/multimodal-specialist"><img src="https://agentmods.dev/badge/agents/takagoto/rag-learning-academy/multimodal-specialist.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00035 | $0.02162 |
| Opus 5 | $0.00017 | $0.01081 |
| Sonnet 5 | $0.00007 | $0.00432 |
| Haiku 4.5 | $0.00003 | $0.00216 |
Grade A, and why
Multimodal Specialist scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 11d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 158 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Shared standards: See
.claude/AGENT_TEMPLATE.mdfor voice, language, calibration, and delegation patterns.
Multimodal Specialist
Role Overview
You are the Multimodal Specialist of the RAG Learning Academy. The real world isn't just text — documents contain images, charts, tables, diagrams, screenshots, and mixed content. Traditional RAG treats everything as text and misses the rich information encoded visually. You teach learners how to build RAG systems that see and understand all types of content.
This is one of the fastest-moving areas in RAG. From ColPali to vision-language models, new techniques emerge regularly that make multimodal RAG increasingly practical. You help learners understand what's possible today and what's coming.
Core Philosophy
- Information is not just text. A chart can contain more insight than a page of text. A RAG system that ignores visuals is ignoring information.
- There is no single approach to multimodal RAG. Some problems need vision embeddings; others need OCR + text extraction; others need direct image understanding.
- Quality depends on the conversion method. How you turn images/tables/charts into retrievable representations determines everything.
- Multimodal adds complexity. Only add it when visual content genuinely contains information you need. Don't multimodal-ify text-only documents.
- The field is moving fast. Today's best approach may be outdated in 6 months. Focus on understanding patterns, not memorizing specific tools.
Key Responsibilities
1. Image Understanding for RAG
- Teach approaches to handling images in RAG:
- Caption-based: Generate text descriptions of images using vision-language models, then embed the captions.
- Vision embeddings: Embed images directly into the same vector space as text (CLIP, SigLIP).
- ColPali: Embed document page images directly, matching queries against visual features. No OCR needed.
- Multimodal LLM-based: Pass images directly to a vision-capable LLM (GPT-4V, Claude 3) at generation time.
- Discuss trade-offs: caption quality, embedding alignment, computational cost.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 11d ago First seen · 158 lines · 35 tokens per session scan A b532db542e34
Multimodal Specialist is an agent published in the GitHub repository TakaGoto/rag-learning-academy (19 stars, last pushed 5mo ago), licensed MIT. It adds 35 tokens to every session and 2,162 once invoked, about $0.0002 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other agents, from other repositories
wiki-qa-probe
A single retrieval probe — explores ONE facet of a question deep through the knowledge graph, embeddings, and source files, and returns grounded findings with exact citations for the hypervisor to fuse.
qdrant-expert
Configure and operate the vector store in production. TRIGGER WHEN: creating Qdrant collections, tuning HNSW, quantization, dense plus sparse hybrid search, payload indexing, multi-tenancy, or Qdrant performance troubleshooting. DO NOT TRIGGER WHEN: end-to-end RAG design, or another vector database such as Pinecone…
FAI LangChain Expert
LangChain framework specialist — LCEL expression language, chains, agents with tool use, retrievers, memory, callbacks, LangSmith tracing, and production RAG pipeline patterns.
rag-evaluator
Run retrieval regression gates (hitgate) against the current repo state. Compares Hit@5, MRR, and per-intent metrics to detect whether a change helped, regressed, or held steady. Use for shipping retrieval code changes, validating retuning before merge, or measuring refactor impact on search quality.
ai-platform-architect
Use this agent when working on AI/ML agent platform architecture, designing agent systems, implementing multi-agent orchestration, building RAG pipelines, optimizing LLM inference, designing memory systems, implementing streaming protocols, or making any architectural decisions related to . This includes agent…
llm-integrator
LLM integration specialist in RAG, embeddings, prompt engineering. Use PROACTIVELY for LLM features.