video-ingest

video-ingest — Video Content Ingestion (capability-routed)

A video carries two separable signals: the spoken transcript (text) and the visual content (on-screen demos, slides, UI). Captions give only the first. So the route is chosen by what the task needs × what engine is available × video length — not a single fixed path. This reuses frontier-digest's capability-over-engine logic and adds on-demand, need-based routing, a Claude-native frame path, output normalization, and a governance cross-check.

FH's increment over a raw video executor (sister asset claude-video, github bradautomates/claude-video; cross-audit note in a private companion store): the executors watch; FH governs — the governor never trusts a multimodal "comprehensive read" at face value, it cross-checks claims against the transcript / a second pass. Same compose pattern as the other FH↔executor pairs.

Triggers

• "ingest this video" · "what does this video show / demonstrate"
• "get the transcript from this YouTube video" · "pull the captions from "
• "summarize what's on screen in this video"
• "video-ingest"

Step 0 — What does the task need? (picks the tier)

• Video understanding (visual: demos, UI walkthroughs, slides, anything not in the spoken words) → a vision path (Tier 1a or 1b below). Captions miss the screen.
• Transcript only (spoken words suffice — a talk, an interview) → go straight to yt-dlp (Tier 3); do not spend a vision engine when text is all that's needed (cost guard).

Step 1 — Capability + length probe, then tier ladder (REUSE the probe — do not re-specify)

Resolve the engine by capability, not name, via frontier-digest's Sidecar Engine Resolution Protocol (plugins/fh-meta/skills/frontier-digest/SKILL_detail.md §Video-Harvest — single source of the probe commands). Length is the 2nd axis (claude-video is sparse past ~10min). The ladder:

• Tier 1a — native multimodal, URL-direct (best for LONG video): an engine that watches the video and returns spoken + visual detail, no frame cap. Verified: Gemini ingests a YouTube URL directly (returns verbatim dialogue AND on-screen detail). Probe for a live route — the direct gemini CLI is EOL (2026-06-18), so probe agy / the Gemini API. Caveat (measured): a YouTube URL works, but an arbitrary (non-YouTube) stream is not reachable headless (needs file upload) — degrade for those.
• Tier 1b — Claude-native frame extraction (Claude-only; SHORT/MEDIUM ≤~10min): when no external multimodal engine is present, do not give up on visual content — ffmpeg extracts frames (auto-scaled, cap ~2fps/100 frames) → Claude's own vision via Read reads them as images (the claude-video pattern). No separate runtime, no Gemini-EOL exposure. Caution: claude-video itself is a 3rd-party executable (brew installs + optional Whisper egress) — if invoking it directly, review first; the ffmpeg+Read mechanism can also be run inline without it.
• Never route video to a coding-agent CLI (codex, a non-multimodal model) — burns tokens, recovers only metadata.
• Tier 3 — yt-dlp transcript (text only; conditional — probe first, never assume):

  yt-dlp --no-update --skip-download --write-auto-subs --sub-langs en \
    --sub-format json3 -o "/tmp/yt_%(id)s.%(ext)s" "<URL>"
  python3 -c "import json,sys,re;d=json.load(open(sys.argv[1]));print(re.sub(r'\s+',' ',' '.join(s.get('utf8','') for e in d['events'] for s in e.get('segs',[]) if s.get('utf8','').strip())).strip())" /tmp/yt_<id>.en.json3
  

  --write-subs (human captions) preferred when present. Keep yt-dlp current (pip install -U yt-dlp; version/impersonation warnings are non-blocking). yt-dlp also supplies the transcript the governor cross-checks a Tier-1 visual read against.
• Auth-gated (members-only / age-gated) → claude-in-chrome (operator's logged-in session). video-ingest surfaces this route as a handoff — it does not drive the browser itself (the mcp__claude-in-chrome__* tools are outside this skill's allowed-tools). Caveat: needs the Anthropic "direct" plan — unavailable on a corp Bedrock laptop; surface the limit, do not loop.
• Unresolvable (no vision engine, yt-dlp probe fails, not auth-fixable) → operator summary remains the path; say so plainly.