NVIDIA AI Blueprint for Video Search and Summarization (VSS) is a reference architecture for applications that use AI to search, analyze, question, and summarize live or recorded video. It is intended for building video analytics agents with vision-language models, language models, retrieval-augmented generation, and NVIDIA NIM microservices. The catalogue entries provide coding-agent rules, skills, commands, agents, instructions, and settings associated with the project.
Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add NVIDIA-AI-Blueprints/video-search-and-summarization --skill rtvi-byom-portinggit clone --depth 1 https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarizationWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/nvidia-ai-blueprints/video-search-and-summarization/rtvi-byom-porting)<a href="https://agentmods.dev/skills/nvidia-ai-blueprints/video-search-and-summarization/rtvi-byom-porting"><img src="https://agentmods.dev/badge/skills/nvidia-ai-blueprints/video-search-and-summarization/rtvi-byom-porting/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/nvidia-ai-blueprints/video-search-and-summarization/rtvi-byom-porting"><img src="https://agentmods.dev/badge/skills/nvidia-ai-blueprints/video-search-and-summarization/rtvi-byom-porting.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00093 | $0.01402 |
| Opus 5.5 | $0.00037 | $0.00561 |
| Sonnet 5.5 | $0.00019 | $0.00280 |
| Haiku 4.5 | $0.00009 | $0.00140 |
Grade A, and why
rtvi-byom-porting scanned grade A with 1 finding against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured yesterday.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Makes network callslowCapability
Not a fault in itself. Listed so you know the mod talks to something, and to what.
the OpenAPI contract and curl examples in How it starts
The opening of the file, as written. The whole thing — 117 lines — stays where its author put it; the contents beside it link to each section on GitHub.
RTVI BYOM Porting
Purpose
Port a VLM into the VSS RT-VLM service without hard-coding one checkpoint or GPU platform. Prefer configuration, then an RTVI adapter or vLLM plugin, and patch vLLM only when the supported extension points cannot express the model.
Use vss-deploy-dense-captioning for ordinary catalog-preset deployment of an
already-wired model (for example cosmos-reason3). Use this skill for a custom
or private checkpoint even when path 1 (configuration only) is enough, and when
architecture, processor, weight mapping, runtime dependencies, or request
adaptation requires repository work.
Model Contract
Before changing code, record:
- immutable model source and revision, license and access requirements;
- architecture, processor, tokenizer, quantization and context length;
- vision-token and frame-sampling assumptions;
- CUDA architecture, memory, vLLM/runtime and custom-kernel requirements.
For a private Hugging Face source, use HF_TOKEN only during authenticated
download/cache population. Never print it, embed it in MODEL_PATH, or bake it
into an image. Prove the cached model starts after the temporary credential is
removed. If remote model code is required, review it first and pair
VLM_TRUST_REMOTE_CODE=true with an exact allowlist entry. For the VSS profile,
set RTVI_VLM_MODEL_PATH_ALLOWLIST; inside the service container this becomes
RTVI_MODEL_PATH_ALLOWLIST. Likewise, the profile input
RTVI_VLM_ALLOW_UNSAFE_MODEL_CONFIG becomes
RTVI_ALLOW_UNSAFE_MODEL_CONFIG; do not enable it without reviewing the
blocked config hooks.
Integration Decision
Stop at the first path that works:
- Configuration only: use
VLM_MODEL_TO_USE=vllm-compatibleand setMODEL_PATHto anngc:URI or a mounted snapshot path. Agit:source is acceptable only for exploration because the develop downloader does not pin Hugging Face revisions; use a revision-pinned mounted snapshot for reproducible evidence. - RTVI adapter: normalize config, processor, request or response behavior
inside
services/rtvi/rt-vlm/while preserving existing model behavior. - Plugin or shim: register architecture and deterministic weight mappings without editing vendored vLLM internals.
- Version-gated patch: patch vLLM only after recording why the first three paths cannot work. Keep it optional and add a focused regression test.
What ships with it
6 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- yesterday Changed · +6 lines · +24 tokens per session 6b23177feed4
- 16d ago Changed 5b773e95a1f8
- 17d ago First seen · 111 lines · 69 tokens per session scan A 71e08d69f36c
rtvi-byom-porting is a skill published in the GitHub repository NVIDIA-AI-Blueprints/video-search-and-summarization (1,932 stars, last pushed today), licensed Apache-2.0. It adds 93 tokens to every session and 1,402 once invoked, about $0.0004 per session on Opus 5.5. A static security scan graded it A with 1 finding (makes network calls). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-30.
Other skills, from other repositories
agent-platform-rag-engine-management
Manage and query Agent Platform RAG Engine Corpora and retrieve grounded contexts using the Google GenAI SDK. Use when listing RAG corpora or files, inspecting a corpus, retrieving contexts, or generating content grounded in a RAG corpus. Do not use for standard database queries (use SQL/Spanner skills), Google…
graphify
Use for any question about a codebase, its architecture, file relationships, or project content — especially when graphify-out/ exists, where the question should be treated as a graphify query first. Turns any input (code, docs, papers, images, videos) into a persistent knowledge graph with god nodes, community…
google-cloud-solution-rag-enterprise-search-gke-sqldb
Discovers requirements, and generates architectural, design, and deployment guidance for a retrieval-augmented generation (RAG)-capable enterprise search system in Google Cloud. Use when users need a vector-enabled SQL database as the store and index for the embedding vectors, an open model and open-source inferencing…
gpt-image-2
A skill for generating or editing images with GPT Image 2 across local, host-provided, or advisory setups.
similarity-search-patterns
Implement efficient similarity search with vector databases. Use when building semantic search, implementing nearest neighbor queries, or optimizing retrieval performance.
kb-retriever
A retrieval and question-answering assistant for a local folder of documents, including Markdown, text, PDFs, and spreadsheets.