NVIDIA AI Blueprint for video search and summarization (VSS) is a GPU-accelerated reference architecture for building video analytics agents with real-time verified alerts, visual Q&A, and automated reporting. The VSS Blueprint uses vision language models (VLMs) such as NVIDIA Cosmos, LLMs such as NVIDIA Nemotron, RAG, and NVIDIA NIMs.
NVIDIA AI Blueprint for Video Search and Summarization (VSS) is a reference architecture for applications that use AI to search, analyze, question, and summarize live or recorded video. It is intended for building video analytics agents with vision-language models, language models, retrieval-augmented generation, and NVIDIA NIM microservices. The catalogue entries provide coding-agent rules, skills, commands, agents, instructions, and settings associated with the project.
This repository also configures its own agents. See what video-search-and-summarization tells them →
NVIDIA-AI-Blueprints/video-search-and-summarization
Skill Cursor
Update the BDD test container image when its dependencies or runtime change. Determines version bump (major vs minor vs patch), rebuilds the Docker image, pushes to GitLab registry, and updates the image tag in starttest.sh. Test source (tests/, features/, scripts/, data/, conftest.py) is bind-mounted from the host…
NVIDIA-AI-Blueprints/video-search-and-summarization
Skill Claude CodeCodex
Use this skill when answering a question about previously analyzed or freshly scoped VSS video, or when reading a stored VSS memory job or record by id, or whenever a question should be answered by running the vss memory introspect command. Route through hot context, agent Markdown notes, vss memory get or vss memory…
NVIDIA-AI-Blueprints/video-search-and-summarization
Skill Claude CodeCodex
Use this skill when deploying standalone RT-VLM dense captioning or calling its REST API (uploads, captions, streams, chat-completions, Kafka). Not for VSS profile deploy or video-search ingestion.
NVIDIA-AI-Blueprints/video-search-and-summarization
Skill Claude CodeCodex
Use this skill when the user wants to deploy, run, debug, tear down, or call the REST API of the RTVI-CV 2D detection / tracking microservice. Trigger when the user says things like 'deploy rtvi-cv', 'start warehouse 2d', 'add a stream', 'check rtvi-cv health', or 'stop the perception container'. Not for VLM…
NVIDIA-AI-Blueprints/video-search-and-summarization
Skill Claude CodeCodex
Use when deploying or operating standalone RTVI-CV-3D / MV3DT multi-camera 3D tracking for calibrated MP4/file inputs and live RTSP streams: missing-calibration handoff to AMC skills, the 4-camera sample dataset, camera config, BEV Fusion, live OSD or saved grid/BEV outputs, bundled brokers, basic external MQTT/Kafka…
NVIDIA-AI-Blueprints/video-search-and-summarization
Skill Claude CodeCodex
Use this skill when deploying, operating, integrating, or customizing the VSS RT-Embed Video Embedding microservice. Covers standalone Docker Compose deployment, the /v1 REST API for text/video embeddings and live streams, Redis/Kafka/OTel integration, troubleshooting, and bring-your-own-model (BYOM) custom embedding…
NVIDIA-AI-Blueprints/video-search-and-summarization
Skill Claude CodeCodex
Use this skill when running or tuning AutoMagicCalib on local MP4s, RTSP, or the bundled sample dataset, or when deploying vss-auto-calibration. Do not use for non-AMC calibration or runtime analytics.
NVIDIA-AI-Blueprints/video-search-and-summarization
Skill Claude CodeCodex
Generates VSS video summary reports with LVS HITL and optional Enterprise RAG document grounding. Trigger when the user asks for a frag/RAG-assisted video report, knowledge-enhanced analysis, or Enterprise RAG context in a video summary.
NVIDIA-AI-Blueprints/video-search-and-summarization
Skill Claude CodeCodex needs its repo
Use when the user asks to deploy, upgrade, or size the VSS warehouse blueprint (2D / 3D / MV3DT) on Kubernetes via Helm — as opposed to Docker Compose, which is covered by vss-build-vision-ai's warehouse reference. Handles GPU-aware NUMSTREAMS capping so the deployment matches what the perception pipeline can actually…
NVIDIA-AI-Blueprints/video-search-and-summarization
Skill Claude CodeCodex
Use this skill to deploy the vss-behavior-analytics service standalone (entrypoint, config-source, optional calibration). Not for the full warehouse deploy.
NVIDIA-AI-Blueprints/video-search-and-summarization
Skill Claude CodeCodex
Use to deploy the vss-video-analytics-api REST service standalone with its Elasticsearch ingest-pipeline, selectable Kafka/Redis stream type, and Kafka-topic readiness gates when Kafka is selected. Not for full warehouse deploy.
NVIDIA-AI-Blueprints/video-search-and-summarization
Skill Claude CodeCodex
Use this skill when producing a VSS analysis report — Mode A per-clip VLM, Mode B incident-range via video-analytics, Mode C SOP compliance via the SOP tools. Not for standalone video summarization, real-time alerts or ad-hoc Q&A.
NVIDIA-AI-Blueprints/video-search-and-summarization
Skill Claude CodeCodex
Use this skill when operating VSS alert workflows — real-time monitoring, Alert-Bridge subscriptions, verification verdicts, on-demand verification, always-on operation, Slack notifications, incident queries, or camera onboarding. Not for non-alert analytics.
NVIDIA-AI-Blueprints/video-search-and-summarization
Skill Claude CodeCodex
Use to drive vss vios for sensor list, timelines, clips, snapshots, and add/delete of video or stream sources, and the VIOS REST API only for what that CLI does not cover (sensor info/status/settings, storage and recorder status, WebRTC, the RTSP proxy, network scan, device settings, bytes to disk, the NvStreamer…
NVIDIA-AI-Blueprints/video-search-and-summarization
Skill Claude CodeCodex
Use this skill for read-only incident, occupancy, speed, place, and analytics-sensor questions through the project-local VSS CLI. Not for live VLM, incident-range narrative reports, deployment, or alert-rule management.
NVIDIA-AI-Blueprints/video-search-and-summarization
Skill Claude CodeCodex
Use this skill when a user wants to search archived VSS video that is already registered in a configured deployment — by natural-language, similarity, attribute, object-ID, or lexical tag query. Not for fresh clip Q&A, live captioning, video summarization, deployment, or source ingestion/deletion.
NVIDIA-AI-Blueprints/video-search-and-summarization
Skill Claude CodeCodex
Use when summarizing a recorded video through HITL-gated LVS, falling back to vss vlm run when LVS is not ready. Not for reports, archive search, or live RTSP captioning.
NVIDIA-AI-Blueprints/video-search-and-summarization
Skill Claude CodeCodex
Add agent-ready vision capabilities — dense captioning, detection, search, alerting, summarization — to an agent or application through a customizable, self-contained vision stack built on the NVIDIA VSS Blueprint. Use this skill when a developer or agent wants to give their app vision: pick capabilities via guided…
NVIDIA-AI-Blueprints/video-search-and-summarization
Skill Claude CodeCodex
Measure whether an RT-VLM configuration change altered caption quality — capture paired baseline and candidate captions for a set of videos, score both against a ground truth with an LLM judge, and emit an accuracy and processing-time table. Use when changing frame selection, decode, or model settings and you need…
NVIDIA-AI-Blueprints/video-search-and-summarization
Skill Claude CodeCodex needs its repo
Benchmark a deployed LVS instance — set up test media, run single-file latency and burst-throughput tests, analyze GPU and latency metrics, and get configuration recommendations to improve performance.
NVIDIA-AI-Blueprints/video-search-and-summarization
Skill Claude CodeCodex
Benchmark video Q&A accuracy and latency of a deployed RT-VLM (Cosmos Reason 3) via vss vlm run, using questions and videos from the DSS vss-devx-base dataset. Replaces the deprecated nat eval / vss-agent QA path. Not for tool-calling or trajectory evaluation, and not for LVS summarization throughput.
NVIDIA-AI-Blueprints/video-search-and-summarization
Skill Codex
Use when adding, debugging, or validating a bring-your-own VLM in VSS RT-VLM, including a custom Hugging Face or NGC checkpoint loaded with configuration only (VLMMODELTOUSE=vllm-compatible), plus adapters, plugins, shims, and model-specific runtime dependencies. Not for ordinary catalog-preset deployment of an…
NVIDIA-AI-Blueprints/video-search-and-summarization
Skill Claude CodeCodex
Profile vLLM and RT-VLM inference to identify and remove GPU-idle gaps, underfilled batches, transfer stalls, serialized multimodal work, scheduler gaps, or KV pressure. Use this skill when GPU utilization is unexpectedly low or bursty, capacity trails another runtime, TTFT or ITL regresses, or multimodal prefill and…
NVIDIA-AI-Blueprints/video-search-and-summarization
Skill Codex needs its repo
Plan, run, and diagnose reproducible RT-VLM GPU performance canaries and benchmarks. Use this skill when running fresh-container stream-capacity, semantic-isolation, latency, throughput, or regression experiments against an RTVI microservices checkout.
At most 3 mods per repository are shown here, and a mod shipped inside a plugin is left to that plugin's page — the rest are on their repository pages: