talking-avatar

Talking Avatar

Create one focused voice conversation with a fixed character portrait whose mouth poses follow the actual remote audio. Accept either a photograph or a character description, generate a canonical portrait plus small mouth sprites, and build the app around OpenAI Realtime.

Coordinate the required skills

• Use imagegen for every canonical portrait or mouth-frame generation/edit. Read its full instructions before generating.
• Use openai-docs and, when applicable, openai-platform-api-key to verify the current Realtime WebRTC endpoint, session schema, model, voices, and credential flow. Never rely on this skill's model examples as current truth.
• Use the available frontend design skill for interface work.
• If .openai/hosting.json exists, follow sites-building and sites-hosting through deployment.
• Do not add backgrounds, wardrobes, role modes, or multiple characters unless the user explicitly asks. The default product is one avatar and one conversation.

Choose the input path

1. Photograph supplied: inspect it with view_image, treat it as an identity reference, and preserve face shape, skin tone, hair, glasses, facial hair, and distinguishing features.
2. Character described: normalize the description into a concrete visual spec. Ask only for a missing detail that materially changes identity, such as photorealistic versus illustrated. Otherwise proceed with tasteful defaults.
3. Establish the character name, concise persona, app name, language, and preferred voice. Infer reasonable defaults when safe.

Read references/image-pipeline.md before creating or editing the avatar assets.

Build the asset set

1. Generate one canonical, front-facing, evenly lit, closed-mouth portrait. Lock its canvas, crop, head position, gaze, pose, clothing, background, and lighting.
2. From that canonical image, generate three separate identity-preserving edits:
   • soft: lips slightly parted;
   • round: a natural rounded vowel;
   • open: a natural medium-open vowel.
3. Repeat the invariants in every edit prompt. Change only the immediate lips and inner mouth. Reject frames with moved eyes, jaw, glasses, head, beard, clothing, or lighting.
4. Crop the smallest practical identical rectangle from all three speech frames. Keep enough surrounding skin for feathering, but do not ship full-frame speech images.
5. Save exactly these project assets by default:
   • public/avatar/avatar-base.jpg
   • public/avatar/mouth-soft.png
   • public/avatar/mouth-round.png
   • public/avatar/mouth-open.png
6. Run scripts/validate_avatar_assets.py --dir <project>/public/avatar and fix all failures.