generate-shots

/project:generate-shots — GENERATOR Expert

You are the GENERATOR expert in the Video Recreation Agent Team.

Your job: Execute the built workflows against ComfyUI, monitor progress, collect outputs.

Prerequisites

• Workflow JSONs in workspace/workflows/
• ComfyUI running and accessible
• Any required input images uploaded

Steps

1. Upload input images to ComfyUI

# For each keyframe that workflows reference
for img in workspace/keyframes/seg_*_first.png workspace/keyframes/seg_*_last.png; do
  filename=$(basename "$img")
  curl -s -X POST "http://${COMFYUI_HOST:-127.0.0.1}:${COMFYUI_PORT:-8188}/upload/image" \
    -F "image=@${img}" \
    -F "subfolder=agent_inputs" \
    -F "type=input"
  echo "Uploaded: $filename"
done

2. Queue workflows to ComfyUI

For API-based models (Kling, LTX — cloud GPU): Queue ALL segments in parallel. They don't compete for local VRAM.

For local models (FLUX, Wan — local GPU): Queue one at a time to avoid OOM.

# Queue a single workflow
python3 -c "
import json, urllib.request

workflow = json.load(open('workspace/workflows/seg_01_workflow.json'))
payload = json.dumps({
    'prompt': workflow,
    'client_id': 'video-recreation-agent'
}).encode()

req = urllib.request.Request(
    'http://${COMFYUI_HOST:-127.0.0.1}:${COMFYUI_PORT:-8188}/prompt',
    data=payload,
    headers={'Content-Type': 'application/json'}
)
resp = urllib.request.urlopen(req)
result = json.loads(resp.read())
print(f'Queued: prompt_id={result[\"prompt_id\"]}')
"

3. Monitor progress via WebSocket

# monitor_progress.py — run as background task
import websocket, json

ws = websocket.WebSocket()
ws.connect(f"ws://{COMFYUI_HOST}:{COMFYUI_PORT}/ws?clientId=video-recreation-agent")

while True:
    msg = json.loads(ws.recv())
    if msg['type'] == 'progress':
        d = msg['data']
        pct = int(d['value'] / d['max'] * 100)
        print(f"Step {d['value']}/{d['max']} ({pct}%)")
    elif msg['type'] == 'executed':
        print(f"Node complete: {msg['data']['node']}")
    elif msg['type'] == 'execution_complete':
        print("DONE")
        break