multimodal-proxy

A conditional skill that sends image, video, or audio work to an external multimodal model, which can understand media as well as text.

In plain words
What is it for?
Use it for tasks such as image analysis, OCR, comparing images, video or audio review, chart interpretation, and interface checks when the main model lacks media support.
Why use it?
It helps when the main model can only read text and therefore cannot inspect screenshots, recordings, charts, or other media directly.

Skill for Claude CodeCodex

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add skills/macgaf/multimodal-proxy-plugin/multimodal-proxy
Any agent
npx skills add macgaf/multimodal-proxy-plugin --skill multimodal-proxy
Clone the repo
git clone --depth 1 https://github.com/macgaf/multimodal-proxy-plugin

Made for: Claude Code, Codex.

Per session 314 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 2,052 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00314 $0.02052
Opus 5 $0.00157 $0.01026
Sonnet 5 $0.00063 $0.00410
Haiku 4.5 $0.00031 $0.00205

Measured yesterday against content hash 7d3044b1e775, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

multimodal-proxy scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured yesterday.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

skills/multimodal-proxy/SKILL.md · 135 lines

How it starts

The opening of the file, as written. The whole thing — 135 lines — stays where its author put it; the contents beside it link to each section on GitHub.

multimodal-proxy 使用规范

通过 multimodal-proxy MCP 的工具,把多模态任务外包给配置好的外部多模态模型,再把文字 结果回填给当前主模型。仅适用场景:已确认主模型是纯文本模型(如 glm-5.2、deepseek-v4 等), 且无法直接理解图像/视频/音频。

激活决策(必须先判断)

  1. 先判断当前主模型是否多模态(能力信息优先于模型名称)

    • 优先读取当前运行时、系统提示或模型元数据声明的输入能力;若标明支持图片、视频或音频, 即视为多模态。
    • 无法获取能力声明时,按下方模型名称清单判断;名称也不在清单中 → 默认视为多模态、不自动激活。
    • 已知多模态模型:GPT-4o / GPT-4.1 / GPT-5(包括 GPT-5.5 及后续支持多模态的版本)、 Claude 3.5+、Gemini 全系、doubao-vision、qwen-vl、glm-4v、Kimi-vision 等
    • 已知纯文本模型:glm-5.2、deepseek-v4-flash/pro、多数纯文本 LLM
    • 不确定 → 不得仅因用户提交媒体或使用触发词而激活本 skill;优先由主模型原生处理, 或在无法接收媒体时向用户说明能力状态不明并询问是否要使用外部代理。
  2. 若主模型是多模态模型

    • 默认由主模型直接处理,不调用 multimodal-proxy
    • 仅当用户显式要求"用外部模型""强制外包""用 doubao 分析"时才调用
  3. 若主模型已确认是纯文本模型,且用户有多模态需求

    • 必须调用 multimodal-proxy 工具;不要尝试让纯文本主模型直接"看"图
  4. 优先级

    • 用户明确要求外部代理 > 已确认纯文本模型 > 主模型原生多模态能力 > 能力不明时不自动代理。

工具

save_clipboard_to_file(剪贴板落盘)

读取系统剪贴板内容,如果是图片则保存为临时 PNG 文件并返回路径。

跨平台支持:macOS(osascript)、Windows(PowerShell)、Linux(wl-paste / xclip)。

用途:绕过纯文本 Agent 主模型对图片输入的硬拦截。用户 Ctrl-V 粘贴截图会被拦截, 但截图仍在系统剪贴板中。本工具从剪贴板读出图片,落盘为文件,返回路径供后续分析。

无参数

返回值

  • 图片:返回文件路径(如 /tmp/mmp-clip-1234567890.png
  • 文本:返回 clipboard_text: <内容>(剪贴板里是文字而非图片)
  • 空:返回提示信息(剪贴板为空或不含图片)

调用时机:当用户说"分析一下我刚截的屏""看看剪贴板里的截图""分析截屏"等, 且当前无法通过 Ctrl-V 粘贴图片时,先调用本工具获取文件路径。

process_multimodal(核心工具)

接收任意数量的图片/视频/音频 + 提示词,按顺序组装成多模态请求交给模型处理。

参数:

  • media(必填):媒体文件列表,每个元素是本地路径或 http(s) URL
    • 图片:jpg/png/gif/webp/bmp/svg
    • 视频:mp4/mov/webm/avi/mkv
    • 音频:mp3/wav/m4a/flac/aac/ogg
    • 可混用多种类型
  • prompts(可选):提示词列表,0~n 条,作为任务指令在媒体之前提交
  • model(可选):覆盖默认模型
  • provider(可选):覆盖默认 provider

用法示例:

  • 单图分析:process_multimodal(["/path/img.png"], ["描述这张图"])
  • 多图对比:process_multimodal(["/a.png", "/b.png"], ["对比这两张图"])
  • OCR 提取:process_multimodal(["/scan.jpg"], ["提取图中所有文字", "保留原始格式"])
  • 图表解读:process_multimodal(["/chart.png"], ["解读这个图表的数据"])

generate_image(图像生成)

根据文字提示词生成图片。需在配置中设置 image_generation 模型。

工作流

场景一:用户给出文件路径

仅在主模型已确认是纯文本模型,或用户明确要求外部代理时:用户直接提供图片/视频/音频文件路径 → 直接调用 process_multimodal 分析。否则由主模型原生处理。

Read the full file on GitHub · 135 lines

Files

What ships with it

1 file beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. yesterday First seen · 135 lines · 314 tokens per session scan A 7d3044b1e775

Subscribe to this mod's changes

multimodal-proxy is a skill published in the GitHub repository macgaf/multimodal-proxy-plugin (2 stars, last pushed 2mo ago), licensed MIT. It adds 314 tokens to every session and 2,052 once invoked, about $0.0016 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.

Related

Other skills, from other repositories

media-ingest

Ingest video, audio, PDF, book, screenshot, and GitHub repo content into the brain. Multi-format handling with entity extraction and backlink propagation. Covers video-ingest, youtube-ingest, and book-ingest subtypes.

garrytan/gbrain · 52 tokens

diagnostic-stem-delivery

Audio production with diagnostic analysis, timecode parsing from documents, and verified export workflow.

HKUDS/OpenSpace · 23 tokens

vox-director

Turn ONE topic into a finished Vox-style paper-collage explainer / ad video, end to end on the Atlas Cloud API + local ffmpeg — script, collage keyframes, motion, voice-over, music, captions, all automated. Use this whenever the user wants a "Vox style" video, a paper/torn-paper collage animation, a "motion collage"…

Alisa0808/vox-director · 236 tokens

seedance-vocab-ja

This skill should be used when the user asks for Japanese Seedance 2.0 prompt wording, Japanese cinematic vocabulary, or translation of camera, lighting, action, VFX, audio, and production terms into Japanese.

Emily2040/seedance-2.0 · 50 tokens

model-compatibility

Model family compatibility matrix covering loaders, resolutions, samplers, CFG, VAE, ControlNet, and LoRA compatibility for SD 1.5, SDXL, Flux, SD3, and video models.

artokun/comfyui-mcp · 47 tokens

comfyui-topology-viz

Turn a network topology into one stylized, AI-generated still image via a self-hosted ComfyUI instance — reuses the same topology model as threejs-network-viz (any of 8 topology-source integrations, or a freeform description). Use when the operator asks for a stylized, flashy, or AI-generated…

automateyournetwork/netclaw · 93 tokens