video-processing

Video Processing

Purpose

Turn long-form video into processed outputs: transcripts, short clips, vertical format with captions, thumbnails. Uses FFmpeg, yt-dlp, and transcription services. All operations via shell commands.

When to Use

• User requests video clipping, transcription, or processing.
• An evaluation or research task involves video content.
• Content creation requires extracting highlights from longer video.
• A surplus compute task involves video analysis.

Prerequisites

Required tools (install if missing):

• ffmpeg and ffprobe — video processing
• yt-dlp — video downloading from 1000+ sites
• Transcription: YouTube auto-subs (free), or Groq/OpenAI Whisper API

Check availability:

which ffmpeg ffprobe yt-dlp 2>/dev/null

Pipeline

Phase 1: Intake

From URL:

yt-dlp --dump-json "URL" 2>/dev/null | python3 -c "
import sys, json
d = json.load(sys.stdin)
print(f'Title: {d[\"title\"]}')
print(f'Duration: {d[\"duration\"]}s')
print(f'Resolution: {d.get(\"width\",\"?\")}x{d.get(\"height\",\"?\")}')
"

From local file:

ffprobe -v quiet -print_format json -show_format -show_streams "file.mp4"

If duration > 2 hours, ask user to specify a segment range.

Phase 2: Download

# Best quality up to 1080p with audio
yt-dlp -f "bv[height<=1080]+ba/b[height<=1080]" -o "source.mp4" "URL"

# Also grab auto-subtitles if available (avoids transcription entirely)
yt-dlp --write-auto-subs --sub-lang en --sub-format json3 \
  --skip-download -o "source" "URL"

If source.en.json3 exists, skip to Phase 4 (transcription already done).

Phase 3: Transcription

Priority order — use the first available:

1. YouTube auto-subs (already downloaded in Phase 2) — free, instant
2. Groq Whisper API — fast cloud, free tier available

   curl -s https://api.groq.com/openai/v1/audio/transcriptions \
     -H "Authorization: Bearer $API_KEY_GROQ" \
     -F [email protected] -F model=whisper-large-v3 \
     -F response_format=verbose_json -F timestamp_granularities[]=word
   

3. OpenAI Whisper API — reliable, paid
4. Local Whisper — if installed, slowest but free

   whisper source.mp4 --model small --output_format json \
     --output_dir . --language en