media-streams

---

name: media-streams description: Twilio Media Streams WebSocket guide. Use when building voice AI with custom STT/TTS, real-time audio processing, bidirectional audio, or integration.

Media Streams Skill

Guide for Twilio Media Streams (<Connect><Stream>) — raw audio WebSocket integration for bring-your-own STT/TTS. Load this skill when building voice AI with custom speech processing, real-time audio analysis, or third-party STT/TTS engines.

---

When to Use Media Streams vs ConversationRelay

Criteria	Media Streams	ConversationRelay
Audio access	Raw mulaw 8kHz frames	No audio — text in/out
STT/TTS	Bring your own	Built-in (Google, Polly)
Protocol complexity	High (binary audio)	Low (JSON text)
Latency control	Full	Twilio-managed
DTMF detection	Bidirectional only	Built-in
Interruption handling	Manual	Built-in
Custom audio processing	Yes (analysis, effects)	No
Time to prototype	Longer	Shorter

Use Media Streams when: you need raw audio access, custom STT/TTS engines, real-time audio analysis (sentiment from tone, speaker diarization), or integration with platforms that expect audio input (Google Cloud Speech, AWS Transcribe, Azure Speech).

Use ConversationRelay when: you want text-based LLM integration with minimal audio plumbing. ConversationRelay handles STT, TTS, interruptions, and barge-in automatically.

---

TwiML Setup

Bidirectional Stream (<Connect><Stream>)

Bidirectional streams allow both receiving and sending audio. The <Connect> verb blocks subsequent TwiML until the stream ends.

const twiml = new Twilio.twiml.VoiceResponse();

// Optional: start background recording before stream
const start = twiml.start();
start.recording({
  recordingStatusCallback: `https://${context.DOMAIN_NAME}/callbacks/call-status`,
  recordingStatusCallbackEvent: 'completed',
});

twiml.say({ voice: 'Polly.Amy' }, 'Connecting you to our assistant.');

const connect = twiml.connect();
connect.stream({
  url: 'wss://your-server.com/audio-stream',
  // Optional: pass custom parameters to the WebSocket
  // name: 'my-stream',
});

return callback(null, twiml);