assemblyai-python-sdk: Instructions file for Claude Code

AssemblyAI Python SDK

Speech-to-text and audio intelligence SDK. Supports pre-recorded transcription, real-time streaming, audio analysis features, and LLM access through AssemblyAI's LLM Gateway.

Quick start

pip install -U assemblyai

import os
import assemblyai as aai

aai.settings.api_key = os.environ["ASSEMBLYAI_API_KEY"]

transcript = aai.Transcriber().transcribe(
    "https://assembly.ai/wildfires.mp3",
    config=aai.TranscriptionConfig(
        speech_models=["universal-3-5-pro", "universal-2"],
        speaker_labels=True,
    ),
)

print(transcript.text)
for utterance in transcript.utterances:
    print(f"Speaker {utterance.speaker}: {utterance.text}")

Auth

Set ASSEMBLYAI_API_KEY as an environment variable, or:

aai.settings.api_key = "your-key"

Key classes

• aai.Transcriber — Transcribe files, URLs, or streams. Methods: transcribe(), transcribe_async(), submit(), list_transcripts()
• aai.AsyncTranscriber — Asyncio counterpart of Transcriber. Same options, every API call a coroutine. Methods: transcribe(), submit(), transcribe_group(), get_by_id(), delete_by_id(), list_transcripts(), upload_file()
• aai.AsyncTranscript — What AsyncTranscriber returns. Same fields as Transcript, coroutine methods
• aai.AsyncClient — Shared httpx.AsyncClient pool for one or more AsyncTranscribers
• assemblyai.prerecorded.v2 — Canonical module for both transcribers, matching the /v2/transcript API. The top-level aai.* names re-export it
• aai.TranscriptionConfig — All transcription options: speech_models, speaker_labels, sentiment_analysis, entity_detection, auto_chapters, content_safety, language_detection, summarization, word_boost, disfluencies
• aai.Transcript — Result object with .text, .status, .utterances, .words, .chapters, .entities, .sentiment_analysis. Methods: get_sentences(), get_paragraphs(), export_subtitles_srt(), export_subtitles_vtt()
• aai.SyncTranscriber — Synchronous pre-recorded transcription: audio in, transcript out, one request (no polling). Methods: transcribe(), transcribe_live(), open_live(), transcribe_async(), warm()
• aai.AsyncSyncTranscriber — Asyncio counterpart of SyncTranscriber. Same input types, config, result, and errors; transcribe(), transcribe_live() and warm() are coroutines; open_live() returns an AsyncLiveSession. Owns an HTTP pool: use async with or await aclose(), or pass an aai.AsyncClient to share one
• aai.SyncTranscriptionConfig — Sync options: model (default universal-3-5-pro), prompt, keyterms_prompt, conversation_context, language_codes, timestamps, sample_rate, channels
• aai.SyncTranscriptResponse — Sync result: .text, .words (SyncWord with confidence always, start/end only when timestamps=True), .confidence, .audio_duration_ms, .session_id, .request_time_ms
• aai.DictationTranscriber — Dictation API: audio uploaded as it is spoken over one live request, transcript (and optional LLM pass) out. Methods: open_live() (push-style DictationLiveSession: write()/close()/result()/abort()), transcribe_live() (iterable, file object, bytes or path), warm(). There is no buffered transcribe(): every entry point posts to /v1/transcribe/live
• aai.AsyncDictationTranscriber — Asyncio counterpart of DictationTranscriber; open_live() returns an AsyncDictationLiveSession. Owns an HTTP pool: use async with or await aclose()
• aai.DictationConfig — Dictation options: sample_rate, channels, language_codes, keyterms_prompt, llm_instruction, rewrite_mode (aai.DictationRewriteMode.fast default | smart). Rejects unknown fields
• aai.DictationResponse — Dictation result: .text, .words (DictationWord with text + confidence), .confidence, .llm_response, .llm_error, .audio_duration_ms, .session_id, .request_time_ms, .sync_time_ms, and .final_text (the LLM rewrite, falling back to .text)
• aai.TextToSpeech — Text-to-Speech HTTP API: one text per request, audio out. Methods: synthesize() (complete audio as a TTSResponse), stream() (a TTSStream context manager yielding bytes as the API generates them), close() / with for a client built from api_key=. Host from aai.settings.tts_base_url (https://tts.us.assemblyai.com; set https://tts.eu.assemblyai.com for the EU zone). Requests use the client's own TTS pool (client.tts_http_client, built when the TextToSpeech is), which keeps idle connections 120 s by default (aai.settings.tts_keepalive_expiry; unset, a shorter keepalive_expiry applies), so consecutive sentences reuse one connection. Every TTS pool shares one process-wide TLS context. A passed client= stays the caller's: client.close() closes both pools
• aai.AsyncTextToSpeech — Asyncio counterpart of TextToSpeech; synthesize() is a coroutine and stream() returns an AsyncTTSStream. Owns an HTTP pool: use async with or await aclose()
• aai.TTSSession — Text-to-Speech WebSocket session: one connection, text in as it is written, audio out. with connects and waits for Begin (session.begin); speak(text) yields one request's audio; generate(text) / flush() -> flush_id / audio(flush_id) for streamed-in text and pipelining (audio() with no id follows the request being written, from another thread); cancel() for barge-in (ends the iteration at once); keep_alive(); on(TTSEvents.X, handler(event)) for Begin, Audio (switches audio from pull to push), FlushDone, WordBoundaries, Cancelled, Termination, Error. Exit sends Terminate and sets session.termination (None after a failure). Refusals and failures raise TTSError (status_code None, request_id = session id). Raw PCM/G.711 only; 2,000 chars per generate unless Begin reports another limits.max_generate_text_length; ~100 chars/s sustained. token= for temporary tokens
• aai.AsyncTTSSession — Asyncio counterpart: async with, coroutine sends, async for chunk in session.speak(text)
• aai.TTSConfig — Text-to-Speech options: voice_id (a library voice such as aai-jane, or a cloned voice by UUID or slug), model_id (assemblyai-tts-1), language, sample_rate, encoding (aai.TTSEncoding: pcm_s16le default | pcm_mulaw | pcm_alaw | mp3), container (aai.TTSContainer: raw default | wav), bit_rate, extend_biometric_consent (owner-only consent renewal of a cloned voice). Rejects unknown fields
• aai.TTSResponse — Text-to-Speech result: .audio (bytes) plus the response headers: .request_id, .region, .model, .voice_id, .language, .encoding, .sample_rate, .container, .bit_rate, .content_type, .voice_expires_at, .billed_characters, .audio_duration_ms. TTSStream.info carries the same fields without audio; audio_duration_ms is None on a stream with audio (0 when the text had nothing to speak)
• aai.TTSError — Raised on any Text-to-Speech refusal: .status_code, .error_code (stable API code; biometric_terms_not_accepted 403 until the account accepts the biometric data terms in the dashboard, auth_unavailable 503 retryable; on a session these arrive as close 3006 and 1011, with status_code None, the close code in .body["close_code"] and .retryable from the API's Error frame), .param, .request_id, .retryable, .retry_after, .errors, .body (the whole envelope)
• assemblyai.streaming.v3.RealTimeTranscriber — Real-time streaming with event-based API (threaded)
• assemblyai.streaming.v3.AsyncRealTimeTranscriber — Asyncio-native counterpart; same options/events
• aai.LLMGateway — LLM Gateway client. Resources: models.list(), chat.completions.create() (incl. stream=True), understanding.create()/.validate()
• aai.AsyncLLMGateway — Asyncio counterpart of LLMGateway. Same resources, every API call a coroutine; owns an HTTP pool (async with or await aclose())
• assemblyai.llm_gateway.v1 — Canonical module for both gateways, matching the gateway's /v1 API. The top-level aai.* names re-export it
• aai.LLMGatewayMessageParam — TypedDict for a chat message; pass plain dicts ({"role": "user", "content": "..."})