mulmoterminal-video-edit

Edit a video

The user points at a video and says what they want in plain words. They do not write ffmpeg commands; you do, in the terminal, and show them the result. Talk to them about their video — the part from 0:12 to 0:40, the subtitles, the title at the start — not about codecs and filters, unless they ask.

What you need

• ffmpeg and ffprobe. Check with ffmpeg -version. If it is missing, say so and give the install command (brew install ffmpeg on a Mac); do not install it yourself.

• For subtitles or captions drawn onto the picture, ffmpeg built with libass. Check: ffmpeg -hide_banner -filters | grep -E ' (subtitles|ass) ' prints two lines when it is.

• For subtitles made from speech, whisper-cli from whisper.cpp (brew install whisper-cpp on a Mac; the same package MulmoTerminal's voice input uses) and a model. Look for one MulmoTerminal has already downloaded, in the workspace's models folder:

  ls "${CLAUDE_CWD:-$HOME/mulmoclaude}/models"/ggml-*.bin
  

  Use the file you find (prefer ggml-large-v3-turbo.bin when there are several) as $model below. If there is none, ask before downloading one, and say how big it is: ggml-large-v3-turbo.bin (about 1.6 GB, the most accurate) or ggml-small.bin (about 490 MB, faster, less accurate). Then put it in that folder, so voice input finds it too:

  models="${CLAUDE_CWD:-$HOME/mulmoclaude}/models" && mkdir -p "$models" &&
  curl -L -o "$models/ggml-large-v3-turbo.bin" https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-large-v3-turbo.bin
  

  (ggml-small.bin the same way.) Then $model is the file you downloaded.

• showInPane, to open the result beside the terminal. In a project directory it exists only when Canvas was switched on before the session started. Without it, give the user the file's path.

The speech is transcribed on this machine. Never send the video or its sound anywhere.

Rules for every edit

• Never change the original. Write a new file in the same folder, named after the original and the edit: talk.mp4 → talk-cut.mp4, talk-subtitled.mp4. If that name is taken, add a number. When the original is outside your working directory, write the new file in your working directory instead, so it can be shown.
• Keep intermediate files in a working directory of their own (work=$(mktemp -d)), and give them plain names (audio.wav, subs.srt), which keeps the filter arguments below free of quoting.
• Encode the result so it plays everywhere, the pane included: H.264 video, AAC sound, yuv420p, and the index at the front of the file.