NVIDIA AI Blueprint for Video Search and Summarization (VSS) is a reference architecture for applications that use AI to search, analyze, question, and summarize live or recorded video. It is intended for building video analytics agents with vision-language models, language models, retrieval-augmented generation, and NVIDIA NIM microservices. The catalogue entries provide coding-agent rules, skills, commands, agents, instructions, and settings associated with the project.
Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add NVIDIA-AI-Blueprints/video-search-and-summarization --skill vss-evaluate-caption-accuracygit clone --depth 1 https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarizationWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/nvidia-ai-blueprints/video-search-and-summarization/vss-evaluate-caption-accuracy)<a href="https://agentmods.dev/skills/nvidia-ai-blueprints/video-search-and-summarization/vss-evaluate-caption-accuracy"><img src="https://agentmods.dev/badge/skills/nvidia-ai-blueprints/video-search-and-summarization/vss-evaluate-caption-accuracy/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/nvidia-ai-blueprints/video-search-and-summarization/vss-evaluate-caption-accuracy"><img src="https://agentmods.dev/badge/skills/nvidia-ai-blueprints/video-search-and-summarization/vss-evaluate-caption-accuracy.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00074 | $0.02186 |
| Opus 5.5 | $0.00030 | $0.00874 |
| Sonnet 5.5 | $0.00015 | $0.00437 |
| Haiku 4.5 | $0.00007 | $0.00219 |
Grade A, and why
vss-evaluate-caption-accuracy scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 17d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 166 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Purpose
Answer one question with evidence: did this RT-VLM change make captions worse?
A configuration change that saves processing time is only useful if caption quality holds. This skill captures captions twice over the same videos — once with the change (HYP) and once without (REF) — scores both against a ground truth using an LLM judge, and reports accuracy delta alongside time saved.
When to Use
- Before/after changing RT-VLM frame selection, decode, or model settings, to prove there is no caption-quality regression
- Quantifying accuracy delta and processing-time saved for a candidate config change
- Verifying a change was behavior-neutral via frame-level provenance, not just matching totals
Not for deploying RT-VLM, general LVS/RTVI benchmarking (see vss-benchmark-video-summarization), or one-off caption spot checks with no paired baseline.
Prerequisites
Everything except the judge runs inside the RT-VLM container. The container
needs a GPU, the model on disk, and the nvdsframeselector DeepStream plugin
(shipped with DeepStream in the RT-VLM image).
| Requirement | Notes |
|---|---|
| Running RT-VLM container | Default name rtvi_vlm-$USER. Start it before running any stage |
| Model weights | Set MODEL_PATH in the deployment .env. Runs here used Qwen3-VL-32B-Instruct; any vLLM-compatible VLM works |
VLM_MODEL_TO_USE=vllm-compatible |
In the deployment .env |
| Source videos | A directory of .mp4 files. Point DEDUP_DIR at it |
OPENAI_API_KEY |
In the deployment .env. Only needed for the gt stage (ground truth is gpt-4.1) |
claude CLI on the host |
The judge shells out to it. It is not installed in the container |
Deployment inside requires-vss |
>=3.2.0,<4.0.0, from this skill's front matter. Not enforced automatically — this skill has no preflight.sh, and it drives RT-VLM's OpenAI-compatible HTTP surface directly rather than a VSS deployment origin, so there is no origin here to check. Verify by hand if a deployment's version is in doubt: python3 <repo>/services/agent/scripts/check_vss_version.py <deployment-origin> --skill <this SKILL.md> (0 = compatible, 3 = incompatible, 1 = indeterminate) |
What ships with it
8 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 17d ago Changed 1b2ad070d613
- 18d ago Changed · +8 lines d3b8b0da3b9a
- 23d ago Changed · +6 lines 66f14a7d3195
- 1mo ago First seen · 152 lines · 74 tokens per session scan A a882c0405b44
vss-evaluate-caption-accuracy is a skill published in the GitHub repository NVIDIA-AI-Blueprints/video-search-and-summarization (1,932 stars, last pushed today), licensed Apache-2.0. It adds 74 tokens to every session and 2,186 once invoked, about $0.0003 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-15.
Other skills, from other repositories
rag-quality-testing
Use this skill when you need evidence-bounded grounding, relevance, completeness, citation support, abstention, and answer-level evidence in RAG outputs; triggers include RAG 质量 and RAG quality.
rag-retrieval-testing
A guide for testing retrieval, the step where a system finds relevant information for a query. It covers query variations, text splitting, filters, ranking, freshness, and approximate measures of recall and precision.
quiz-generator
Generate educational quizzes from topics, learning goals, or provided documents. Supports multiple question types, difficulty levels, answer keys, explanations, and study-focused assessment. Use when the user wants practice questions, review quizzes, mock tests, self-checks, or exam prep.
exam2knowledge
An exam-analysis skill that works backward from exam or practice questions to identify frequently tested topics and reusable solution patterns.
ad-campaign-analyzer
Analyze cross-channel campaign data, quantify uncertainty, and propose evidence-labeled budget tests without overstating causality.
agent-evaluation
Evaluate agent behavior with versioned cases and explicit verifiers. Use when comparing agent or prompt changes, reproducing failures, or running agent regression tests.