voice-ai

Overview

Voice AI covers TTS, STT, voice cloning, and real-time conversational agents. Integrates ElevenLabs, OpenAI TTS, Whisper, and Vapi/Bland/Retell.

Capabilities

• Text-to-speech (ElevenLabs, OpenAI TTS)
• Audio transcription (Whisper, Deepgram)
• Voice cloning from samples
• Real-time voice agents (Vapi, Bland, Retell)
• Audio processing (pydub)

When to Use

Trigger phrases:

• "voice ai"

• "Voice AI — text-to-speech (ElevenLabs, OpenAI TTS), speech-to-text (Whisper), vo"

• Voiceovers for videos/podcasts

• Voice-based customer support

• Transcribing meetings/interviews

• Audiobook generation

When NOT to Use

• Task is about content strategy, not creation (use strategy skills)
• Task is about content distribution (use distribution skills)
• You need to analyze content performance (use analytics skills)
• Task is about content moderation (use moderation tools)
• You don't have content guidelines
• Task requires domain expertise (consult experts)

Pseudo Code

The voice-ai workflow follows a standard pipeline pattern.

Core flow:

# voice-ai primary flow
input = prepare(raw_data)
result = process(input, config={agents, cloning, elevenlabs, openai, real})
validate(result)
deliver(result)

Error handling:

on error:
  log(error_details)
  retry_with_backoff(max=3)
  if still_failing: alert_and_escalate()

Core Workflow

# voice-ai primary flow
input = prepare(raw_data)
result = process(input, config={agents, cloning, elevenlabs, openai, real})
validate(result)
deliver(result)

Error Handling

on error:
  log(error_details)
  retry_with_backoff(max=3)
  if still_failing: alert_and_escalate()

OpenAI TTS

from openai import OpenAI
client = OpenAI()
resp = client.audio.speech.create(model="tts-1-hd", voice="nova", input="Hello!")
resp.stream_to_file("output.mp3")

ElevenLabs Clone

import requests
resp = requests.post("https://api.elevenlabs.io/v1/voices/add",
    headers={"xi-api-key": KEY}, files={"files": [open("sample.mp3","rb")]}, data={"name":"Clone"})
voice_id = resp.json()["voice_id"]
resp = requests.post(f"https://api.elevenlabs.io/v1/text-to-speech/{voice_id}",
    headers={"xi-api-key": KEY}, json={"text":"Hello!","model_id":"eleven_multilingual_v2"})