hyperframes-media

HyperFrames media

Use fish-audio-api for narration and media-use to ingest reviewed local assets. This package supplies workflows and code; users supply accounts, permitted voices, music, sound effects, images and video.

Requirements and scope

Node >=22.20 and FFmpeg/ffprobe must be on PATH. Read requirements before generation. The engine loads the video project's .env; existing environment values win. Fish uses its official API with FISH_API_KEY and FISH_REFERENCE_ID. Word timing uses OpenAI whisper-1 with OPENAI_API_KEY, or an explicitly configured OpenAI-compatible WHISPER_API.

The shared engine performs one POST per requested narration line and one transcription POST. Neither is automatically retried. It preflights both API configurations before creating voice audio. A whole --only tts rerun creates narration again; use the scaffolded paragraph pipeline for cached/surgical revisions. All generated audio needs listening review.

Shared engine

node <MEDIA_DIR>/scripts/audio.mjs --request audio_request.json --hyperframes . --out audio_meta.json --only tts
node <MEDIA_DIR>/scripts/audio.mjs --request audio_request.json --hyperframes . --out audio_meta.json --only bgm,sfx

The second command needs no API credentials. It preserves and validates any existing voice metadata. Example request (all files shown must be supplied by the user):

{
  "lang": "zh",
  "fish": {
    "transport": "official-api",
    "request": { "normalize": true, "prosody": { "speed": 1 } }
  },
  "lines": [
    { "id": "s0", "text": "这是一段待审听的旁白。", "sfx": [
      { "name": "confirmation", "path": "library/confirmation.wav", "offset_s": 0.2 }
    ] }
  ],
  "bgm": { "path": "library/underscore.wav", "source": "reviewed-local", "volume": 0.18 }
}

fish.reference_id and fish.model are optional project overrides; no personal voice is supplied. fish.request uses official API fields, such as top-level temperature, top_p, normalize and prosody. The adapter owns text, reference ID and WAV format. It rejects credential-like fields and website backend configuration.