Automatic Speech Recognition (ASR)

Automatic Speech Recognition (ASR)

Overview

After speaker diarization, you need to transcribe each speech segment to text. Whisper is the current state-of-the-art for ASR, with multiple model sizes offering different trade-offs between accuracy and speed.

When to Use

• After speaker diarization is complete
• Need to generate speaker-labeled transcripts
• Creating subtitles from audio segments
• Converting speech segments to text

Whisper Model Selection

Model Size Comparison

Model	Size	Speed	Accuracy	Best For
tiny	39M	Fastest	Lowest	Quick testing, low accuracy needs
base	74M	Fast	Low	Fast processing with moderate accuracy
small	244M	Medium	Good	Recommended balance
medium	769M	Slow	Very Good	High accuracy needs
large-v3	1550M	Slowest	Best	Maximum accuracy

Recommended: Use small or large-v3

For best accuracy (recommended for this task):

import whisper

model = whisper.load_model("large-v3")  # Best accuracy
result = model.transcribe(audio_path)

For balanced performance:

import whisper

model = whisper.load_model("small")  # Good balance
result = model.transcribe(audio_path)

Faster-Whisper (Optimized Alternative)

For faster processing with similar accuracy, use faster-whisper:

from faster_whisper import WhisperModel

# Use small model with CPU int8 quantization
model = WhisperModel("small", device="cpu", compute_type="int8")

# Transcribe
segments, info = model.transcribe(audio_path, beam_size=5)

# Process segments
for segment in segments:
    print(f"[{segment.start:.2f}s -> {segment.end:.2f}s] {segment.text}")

Advantages:

• Faster than standard Whisper
• Lower memory usage with quantization
• Similar accuracy to standard Whisper

Aligning Transcriptions with Diarization Segments

After diarization, you need to map Whisper transcriptions to speaker segments: