Speaker Clustering Methods

Speaker Clustering Methods

Overview

After extracting speaker embeddings from audio segments, you need to cluster them to identify unique speakers. Different clustering methods have different strengths.

When to Use

• After extracting speaker embeddings from VAD segments
• Need to group similar speakers together
• Determining number of speakers automatically or manually

Available Clustering Methods

1. Hierarchical Clustering (Recommended for Auto-tuning)

Best for: Automatically determining number of speakers, flexible threshold tuning

from scipy.cluster.hierarchy import linkage, fcluster
from scipy.spatial.distance import pdist
import numpy as np

# Prepare embeddings
embeddings_array = np.array(embeddings_list)
n_segments = len(embeddings_array)

# Compute distance matrix
distances = pdist(embeddings_array, metric='cosine')

# Create linkage matrix
linkage_matrix = linkage(distances, method='average')

# Auto-tune threshold to get reasonable speaker count
min_speakers = 2
max_speakers = max(2, min(10, n_segments // 2))

threshold = 0.7
labels = fcluster(linkage_matrix, t=threshold, criterion='distance')
n_speakers = len(set(labels))

# Adjust threshold if needed
if n_speakers > max_speakers:
    for t in [0.8, 0.9, 1.0, 1.1, 1.2]:
        labels = fcluster(linkage_matrix, t=t, criterion='distance')
        n_speakers = len(set(labels))
        if n_speakers <= max_speakers:
            threshold = t
            break
elif n_speakers < min_speakers:
    for t in [0.6, 0.5, 0.4]:
        labels = fcluster(linkage_matrix, t=t, criterion='distance')
        n_speakers = len(set(labels))
        if n_speakers >= min_speakers:
            threshold = t
            break

print(f"Selected: t={threshold}, {n_speakers} speakers")

Advantages:

• Automatically determines speaker count
• Flexible threshold tuning
• Good for unknown number of speakers
• Can visualize dendrogram

2. KMeans Clustering

Best for: Known number of speakers, fast clustering