watch

watch

Purpose

Look at a rendered video and report what is actually on screen and in the audio. By default it watches the entire video and considers visuals, voiceover, music, and sound effects. Callers can narrow the scope (specific timestamp ranges, lower frame rate, disable audio tracks) when they want a cheaper or more focused pass.

This is the observation primitive that watch-and-refine calls before deciding what to fix. Other review and editing skills can call it directly.

Inputs

• video — path to a local video file. Required.
• ranges — optional list of [start, end] timestamps to watch. Accepts SS, MM:SS, or HH:MM:SS. Defaults to the whole video.
• fps — frame sampling rate. Defaults to auto by duration (≤30s → 1–2 fps, 30s–1min → ~1 fps, 1–3min → ~0.5 fps, 3–10min → ~0.25 fps). Hard cap 2 fps.
• max_frames — hard cap on total frames sampled across all ranges. Default 100.
• resolution — frame width in px. Default 512. Bump to 1024 only when on-screen text legibility matters.
• include_voice — bool, default true. Transcribe spoken VO/dialogue.
• include_music — bool, default true. Describe music presence, swells, drops, gain relative to VO.
• include_sfx — bool, default true. Note sound effects, foley, transition stingers.
• focus — optional free-text prompt describing what to pay attention to (e.g. "watch the end card", "judge cut timing on the beat drop").

If all three audio flags are false, the skill runs frames-only and notes this in the manifest.

Workflow

1. Validate video exists and is readable. Probe duration with ffprobe.
2. Resolve ranges: if empty, use [0, duration]. Reject ranges outside the file duration.
3. Resolve fps: use caller value if provided, else auto-scale from total resolved range duration. Clamp at 2 fps.
4. Allocate the max_frames budget across ranges proportionally to range duration.
5. Extract frames with ffmpeg into frames/ at the resolved fps and resolution.
6. If any audio flag is true, extract the audio for the resolved ranges to a working WAV. Run a transcript pass when include_voice=true; degrade to frames-only and flag a warning if no Whisper backend is available.
7. Compose observation.md — a timestamp-keyed report. Each entry references the frame paths visible during that window plus any transcript line and audio notes (music/SFX) for the same window. If focus is set, lead each entry with what was observed about that focus.
8. Write manifest.json capturing the resolved inputs (ranges, fps, frame count, audio flags) and output paths.