NVIDIA AI Blueprint for Video Search and Summarization (VSS) is a reference architecture for applications that use AI to search, analyze, question, and summarize live or recorded video. It is intended for building video analytics agents with vision-language models, language models, retrieval-augmented generation, and NVIDIA NIM microservices. The catalogue entries provide coding-agent rules, skills, commands, agents, instructions, and settings associated with the project.
Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add NVIDIA-AI-Blueprints/video-search-and-summarization --skill vss-deploy-dense-captioninggit clone --depth 1 https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarizationWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/nvidia-ai-blueprints/video-search-and-summarization/vss-deploy-dense-captioning)<a href="https://agentmods.dev/skills/nvidia-ai-blueprints/video-search-and-summarization/vss-deploy-dense-captioning"><img src="https://agentmods.dev/badge/skills/nvidia-ai-blueprints/video-search-and-summarization/vss-deploy-dense-captioning/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/nvidia-ai-blueprints/video-search-and-summarization/vss-deploy-dense-captioning"><img src="https://agentmods.dev/badge/skills/nvidia-ai-blueprints/video-search-and-summarization/vss-deploy-dense-captioning.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00051 | $0.03768 |
| Opus 5.5 | $0.00020 | $0.01507 |
| Sonnet 5.5 | $0.00010 | $0.00754 |
| Haiku 4.5 | $0.00005 | $0.00377 |
Grade B, and why
vss-deploy-dense-captioning scanned grade B with 2 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 12d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Asks for rootmediumPrivilege escalation
A mod that escalates privileges can change anything on the machine, not only the project.
- Docker group access and `sudo` are effectively root-level privileges. Use the non-interactive `sudo -n` guard in the deploy reference and stop for host-owner action when passwordless sudo is unavailable. Makes network callslowCapability
Not a fault in itself. Listed so you know the mod talks to something, and to what.
- `curl`, `jq`, the current VSS checkout, and a writable directory for standalone env/state. How it starts
The opening of the file, as written. The whole thing — 279 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Purpose
Stand up the RT-VLM dense-captioning microservice on its own and exercise every endpoint it exposes (file upload, generate_captions, stream add/delete, chat-completions, Kafka topics).
When to Use
- Deploy standalone RT-VLM dense captioning when no full VSS profile is running
- Generate dense captions for an uploaded file or RTSP stream via the RT-VLM REST API
- Manage RT-VLM stream lifecycle (add/get-info/delete) or call chat-completions against it
- Wire RT-VLM to Kafka/ELK/VIOS or sibling model services
Not for deploying a full VSS profile (use vss-build-vision-ai) or ingesting video for search (video-search ingestion is a separate path).
Prerequisites
For standalone RT-VLM deployment:
- Docker, Docker Compose, NVIDIA Container Toolkit, and a visible GPU.
- NGC registry credentials in
$NGC_CLI_API_KEYfordocker login nvcr.io, image pulls, and local NGC model/artifact downloads. curl,jq, the current VSS checkout, and a writable directory for standalone env/state.
For API calls against an existing service:
- Running RT-VLM service reachable at
$BASE_URL. - Bearer token in
$RTVI_VLM_API_KEYor$NGC_CLI_API_KEY, depending on how the service was configured.
For full VSS profile deployment:
- Use
../../vss-build-vision-ai/SKILL.md; this skill does not deploy full VSS profiles.
Instructions
Follow the routing tables and step-by-step workflows below. Each section that ends in workflow, quick start, or flow is intended to be executed top-to-bottom. Detailed reference material lives in references/; execute the documented workflows directly unless a future revision names a concrete helper.
Examples
Worked end-to-end examples are kept under evals/ (each *.json manifest contains a runnable scenario) and inline in the per-workflow curl blocks below. Run a Tier-3 evaluation with nv-base validate <this-skill-dir> --agent-eval to replay them.
Limitations
- Requires either a standalone RT-VLM service deployed via this skill or an existing RT-VLM service reachable from the caller.
- NGC-hosted models and NIMs may be subject to rate-limits, GPU memory requirements, and license restrictions.
- Concurrency, GPU memory, and storage limits depend on the host hardware and the profile's compose file.
- Keep
NGC_CLI_API_KEY,RTVI_VLM_API_KEY, andrtvi-vlm.envfiles out of git and out of logs; do not echo credential values or include them in final responses. - Docker group access and
sudoare effectively root-level privileges. Use the non-interactivesudo -nguard in the deploy reference and stop for host-owner action when passwordless sudo is unavailable.
What ships with it
11 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
- BENCHMARK.md 3.5 KB
- evals/.gitkeep 0 B
- evals/alerts_profile_api.json 6.0 KB
- evals/evals.json 2.9 KB
- evals/standalone_api.json 7.4 KB
- references/api-surface-26.05.md 6.1 KB
- references/deploy-rt-vlm-service.md 39 KB
- references/integrate-rt-vlm.md 27 KB
- references/kafka-workflows.md 15 KB
- scripts/tests/test_deploy_examples.py 3.1 KB runs code
- skill-card.md 4.3 KB
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 12d ago Changed · +5 lines 2c31d7b6c4f8
- 28d ago Changed 3b1c096ef477
- 29d ago Changed · +9 lines 2f76df9b26cf
- 1mo ago Changed f2a57201e15a
- 1mo ago Changed · +5 lines c3b693403c3a
- 1mo ago First seen · 260 lines · 51 tokens per session scan B 93d784c4a371
vss-deploy-dense-captioning is a skill published in the GitHub repository NVIDIA-AI-Blueprints/video-search-and-summarization (1,953 stars, last pushed today), licensed Apache-2.0. It adds 51 tokens to every session and 3,768 once invoked, about $0.0002 per session on Opus 5.5. A static security scan graded it B with 2 findings (asks for root, makes network calls). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
apigee-agent-gateway-governance
Stands up an Apigee API proxy that a Google Cloud Agent Gateway calls as an ext-proc (Service Extension) callout on an agent's LLM traffic, to meter token usage per model, extract caller identity, and apply Model Armor prompt screening or a semantic cache. Use when wiring Apigee in as an AI gateway behind an Agent…
agent-platform-rag-engine-management
Manage and query Agent Platform RAG Engine Corpora and retrieve grounded contexts using the Google GenAI SDK. Use when listing RAG corpora or files, inspecting a corpus, retrieving contexts, or generating content grounded in a RAG corpus. Do not use for standard database queries (use SQL/Spanner skills), Google…
graphify
Use for any question about a codebase, its architecture, file relationships, or project content — especially when graphify-out/ exists, where the question should be treated as a graphify query first. Turns any input (code, docs, papers, images, videos) into a persistent knowledge graph with god nodes, community…
google-cloud-solution-rag-enterprise-search-gke-sqldb
Discovers requirements, and generates architectural, design, and deployment guidance for a retrieval-augmented generation (RAG)-capable enterprise search system in Google Cloud. Use when users need a vector-enabled SQL database as the store and index for the embedding vectors, an open model and open-source inferencing…
gpt-image-2
A skill for generating or editing images with GPT Image 2 across local, host-provided, or advisory setups.
kb-retriever
A retrieval and question-answering assistant for a local folder of documents, including Markdown, text, PDFs, and spreadsheets.