clean-cut

clean-cut — the step-1 cut pipeline

Turn a project's raw clips (videos/video-N/DJI_*.MP4) into a clean master + edited-transcript.json (the word-level timing spine every later step anchors to). The single source of truth is videos/video-N/work/analysis/cuts.json — shared by Claude and the editor UI. Every tool lives in tools/ and takes the project dir as its first arg.

You (Claude) author the cuts by reading the transcript. No separate LLM call. The tools handle audio, encoding, QA, and the editor; the judgment — what is a retake, a false start, filler, or fluff — is yours.

Pipeline (run in order)

Let P = the project (e.g. video-1). Clip id = a short handle (0233); every artifact for a clip is named by that id (0233.wav, 0233.json). The raw MP4 path is stored per-clip in cuts.json as file.

1. Extract 16 kHz mono WAV per clip → P/work/audio/<id>.wav (used for transcription + the RMS noise-floor / snap-to-audio tails). Not scripted — run ffmpeg per clip: ffmpeg -i videos/video-1/DJI_...0233_D.MP4 -vn -ac 1 -ar 16000 videos/video-1/work/audio/0233.wav

2. Draft this video's keyterms → P/work/keyterms.txt (do this before transcribing). Keyterms bias the recognizer toward this video's proper nouns / product / tech names so they aren't mangled (e.g. "Seedream" not "sea dream", "Cloudflare" not "cloud flare"). Accuracy here is load-bearing: the transcript text drives cut decisions AND /make-tsx greps it for phrases to time beats — a garbled term breaks both. From the video's topic/title, list the ~10–40 likely brand names, tools, tech, and jargon, one per line (blank lines and # comments ignored). This is per-video — never hardcode terms in transcribe.py. If you skip the file, transcription still runs (empty fallback), just with more errors on specialty words. The shape is one term per line:

   # tools + brands named in this video
   Claude Code
   Remotion
   AssemblyAI
   ElevenLabs
   Cloudflare
   

3. Transcribe (needs ASSEMBLYAI_API_KEY in .env; verbatim, keeps fillers; auto-loads work/keyterms.txt): python tools/transcribe.py P → P/work/transcripts/<id>.json. --clips 0233 for one, --force to redo. It prints how many keyterms it loaded — a "none" line means you haven't drafted them.

4. Readable take view for analysis: python tools/format_transcript.py P → P/work/analysis/takes-<id>.txt (segments on >0.8s gaps, fillers tagged inline with timestamps).

5. Author cuts.json (see schema below) by reading takes-*.txt: mark every span as a keep or a categorized cut, add fluff suggestions and judgment-call flags.

6. QA + review docs: python tools/analyze_cut.py P [--style tight] → qa-report.md (internal dead-air, clipped-tail risks, tiny fragments, fluff, hard entries at cut joins, ghost speech = untranscribed energy riding inside a keep, low-confidence kept tokens). Ghost/hard-entry checks exist because a transcript diff CANNOT see a mistimed token (clipped word onset) or an untranscribed false start ("and it—") that survives the cut — only energy-vs-token cross-checks catch them (a careful listen caught both before these checks existed). python tools/make_review.py P → review.md (per-clip keep/cut table + estimated length per style).

7. Editor proxy (once): python tools/make_proxy.py P → P/work/editor/{proxy.mp4, waveform.png, manifest.json} (720p concat of raw clips + per-clip offsets).

8. Previews (render BOTH, user picks): python tools/render_cuts.py P --style tight --mode preview and --style natural → P/output/preview-<style>.mp4 (720p h264_nvenc). 8.5. Machine verification of the render (MANDATORY after every preview render, before showing the user). Extract the preview's WAV → transcribe.py P --clips preview --force → python tools/verify_cut.py P → verify-report.md. A second ASR pass over the RENDER, diffed against the intended kept tokens: EXTRA words = untranscribed ghosts that rode along (false starts glued to word tails — invisible to the raw transcript, and energy heuristics can't tell them from word releases); MISSING words = clipped/dropped; plus interior-pause anomalies and low-confidence rendered tokens. Born in testing: a mistimed ASR token clipped a word onset ('slash dot env' → '...env') and a ghost 'and it—' survived to the render; a careful listen caught both, now these tools do. Treat every finding as "listen here": explain each one or fix it — don't declare the cut good while the report has unexplained lines.

9. USER AUDIT — this is a hard gate, same as the plan step. Open the editor: python tools/editor/server.py P → http://localhost:8765. User drags keep/cut edges, adds cuts (I/O + C), compares raw vs edited playback; Save rewrites cuts.json (backup to work/analysis/backups/, appended to changes.log); Render button re-runs a preview. Iterate until approved.

10. Final master: python tools/render_cuts.py P --style <chosen> --mode final → P/output/master-<style>.mp4 (4K60 10-bit hevc_nvenc). Two MANDATORY post-render steps:

• A/V duration gate: ffprobe -show_entries stream=duration on v:0 vs a:0 — they MUST be equal. verify_cut's A/V budget GROWS along the timeline (±2s by mid-video) and masks a real accumulating drift; the equal-duration check is the definitive one. (See the drift note under Notes.)
• Playable/handoff transcode: the 10-bit HEVC master won't play in most players or the IDE preview, and the HEVC final stamps frames ~0.1% fast on 59.94fps footage. Produce an 8-bit H.264 that fixes both by re-timing to true CFR: ffmpeg -r <src_fps> -i master-<style>.mp4 -c:v libx264 -crf 19 -pix_fmt yuv420p -c:a aac master-<style>-h264.mp4 — the source fps BEFORE -i re-stamps every frame (no frame loss) so v:0==a:0. This is the file the user reviews AND the comp-native source downstream steps use.

1. Handoff spine — edited-transcript.json: word times in the FINAL master timeline. Simplest robust path (what video-1 used): extract the master's WAV and transcribe.py it, then normalize to {words:[{text,start,end}...]} in ms. (A cuts.json time-remapper is the planned alternative.) This file is what /make-tsx reads to sync visuals to speech.