federated-learning

Federated Learning

Federated learning (FL) trains machine learning models across decentralized data sources without centralizing raw data — devices or silos compute local updates, and only model parameters (gradients or weights) are aggregated by a coordinator. This skill covers the algorithmic foundations (aggregation strategies, privacy guarantees, communication efficiency), the production frameworks (Flower, PySyft), and the threat models (poisoning, inference attacks) that determine whether a FL deployment is actually privacy-preserving or merely privacy-theater.

Core Concepts

FedAvg and Its Variants

FedAvg (McMahan et al. 2017) is the canonical aggregation algorithm. Each round: (1) server samples K clients from N total; (2) each selected client downloads the global model, runs E epochs of SGD on its local data, uploads the updated weights; (3) server computes a weighted average of updates, weights proportional to local dataset size.

# Server-side aggregation (simplified)
def fedavg_aggregate(client_updates):
    total_samples = sum(n for _, n in client_updates)
    aggregated = {}
    for params, n_samples in client_updates:
        weight = n_samples / total_samples
        for key, val in params.items():
            aggregated[key] = aggregated.get(key, 0) + weight * val
    return aggregated

Hyperparameters: C (client fraction per round, 0.1 is typical for large deployments), E (local epochs, 1-5), B (local batch size). High E causes client drift — local models overfit to local data and diverge from each other, making aggregation less effective. This is the core challenge of non-IID data.

FedProx adds a proximal term to the local objective to limit drift: minimize F_k(w) + (μ/2)||w - w_global||². The μ parameter (0.001–1.0) controls how close local updates stay to the global model. FedProx degenerates to FedAvg when μ=0. Critical for heterogeneous (non-IID) settings.

SCAFFOLD uses control variates to correct client drift without restricting local optimization. Two extra vectors per client (c_i, c) correct the gradient direction. Better convergence than FedProx theoretically but doubles communication cost (send c_i update each round).