precise-bilingual-subtitle

Precise Bilingual Subtitle Production

Produce hardcoded bilingual (English + Chinese) video subtitles with Whisper word-level timing, not YouTube's unreliable auto-captions. Supports full visual customization: font, size, color, stroke, and bottom margin.

When to use this skill

• YouTube auto-captions have visible timing drift (the most common failure)
• User wants yellow, large, or otherwise custom-colored subtitles
• Subtitle timing must be frame-accurate and match lip movements
• Chinese YouTube subtitles hit HTTP 429 and cannot be downloaded
• User wants English-on-top / Chinese-on-bottom bilingual layout

Core insight: why YouTube auto-captions fail

YouTube auto-generated captions are aligned to the entire video stream at once, not word-by-word. Result: systematic offset, drift over time, and poor sync with speech. Local Whisper transcribes with per-word timestamps from the extracted audio, producing timing that is reliable enough for professional delivery.

Proven workflow

Phase 1: Extract audio and transcribe with Whisper

# 1. Extract mono 16kHz audio from the source video
ffmpeg -y -i source.mp4 -vn -ac 1 -ar 16000 audio.wav

# 2. Run Whisper with word-level timing (turbo model, ~30s for 5-min clip)
whisper audio.wav --model turbo --language en --task transcribe \
  --output_format srt --output_dir .

Whisper's SRT output has per-word-level timestamps — each entry is 1-3 words with precise start/end times. This is the foundation of accurate subtitle sync.

Phase 2: Group raw fragments into readable subtitle chunks

Raw Whisper output has hundreds of tiny fragments. Group them into readable subtitle blocks:

import re

def group_whisper_srt(srt_path, min_words=12):
    """Group raw Whisper fragments into readable subtitle chunks."""
    # Parse SRT...
    grouped = []
    buf, start, end = [], None, None
    
    for item in raw_items:
        if start is None:
            start = item['start']
        end = item['end']
        buf.append(item['text'])
        
        wc = len(' '.join(buf).split())
        # Group by word count OR sentence-ending punctuation
        if wc >= min_words or item['text'].strip().endswith(('.', '?', '!', ':', '."')):
            merged = ' '.join(buf)
            merged = re.sub(r'\s+([,.;?!])', r'\1', merged)
            grouped.append({'start': start, 'end': end, 'en': merged})
            buf, start, end = [], None, None
    
    # Don't forget remaining
    if buf:
        grouped.append({'start': start, 'end': end, 'en': ' '.join(buf)})
    
    return grouped