diversity-and-population-management

Diversity and Population Management

You are an expert in population management for evolutionary and population-based metaheuristics. This skill is the reference catalog for everything that keeps a population useful: diversity measurement (entropy-based and distance-based), preservation mechanisms (fitness sharing, clearing, crowding, restricted tournament selection, duplicate elimination, mating restrictions), restoration mechanisms (restart policies, random immigrants), and adaptive parameter control wired to diversity signals. For every mechanism it gives when to use it, a numpy implementation, a complexity note, and the algorithms and problem types it fits. Use the measure-preserve-restore framework below to choose mechanisms instead of stacking them blindly.

Initial Assessment

Establish these facts before recommending or writing any diversity machinery:

• Decide the actual goal first. One best solution, or several distinct high-quality solutions (multimodal search, alternatives for a decision maker)? Niching proper (sharing, clearing, crowding) exists for the second goal. For the first goal, cheaper tools — duplicate elimination, sane selection pressure, restarts — usually suffice and cost less.
• Confirm the diagnosis before treating it. "The GA stopped improving" has at least three causes: lost diversity (entropy collapsed), a genuinely hard landscape (diversity fine, no better solutions nearby), or broken variation operators. Plot best fitness AND a genotype diversity measure over generations before choosing a fix. If diversity is healthy and search still stalls, this skill is the wrong lever — see fitness-landscape-analysis.
• Identify the pressure source. Diversity loss is caused by selection and replacement: tournament size, elitism strength, steady-state replacement of the worst. Takeover time under tournament selection is roughly $\log N / \log t$ generations (Goldberg & Deb 1991, "A Comparative Analysis of Selection Schemes"). Reducing pressure at the source (see selection-and-replacement-strategies) is often cheaper than adding a preservation mechanism on top.
• Fix the encoding and its distance. Every mechanism here needs a distance or a frequency count on genotypes. Hamming works for binary/integer strings; permutations need a deliberate choice (positional Hamming vs edge-based); real vectors use Euclidean. Symmetric encodings (rotations/reflections of a tour, relabelable groups) inflate distances between identical solutions — canonicalize first or measure in phenotype space.
• Establish the evaluation cost. Restarts and immigrants re-spend evaluations; with an expensive objective, prefer preservation mechanisms that waste nothing (dedup, RTS, crowding). With cheap evaluations, restarts are often the best value per line of code.
• Establish the population size and budget. Small populations ($N \le 50$) drift to uniformity even without selection pressure; no sharing parameter rescues that. Large populations make $O(N^2)$ sharing the bottleneck — budget the per-generation overhead against one fitness evaluation.
• Check the algorithmic context. A canonical GA, a memetic algorithm (local search collapses diversity much faster), an EDA (diversity lives in model variance — see estimation-of-distribution-algorithms), or scatter search (the reference set already encodes a diversity rule — see scatter-search-path-relinking)? The host determines where the mechanism plugs in.
• Reproducibility. Every stochastic component takes an explicit np.random.Generator. Restart-policy comparisons without fixed seeds, equal evaluation budgets, and $\ge 10$ repetitions are noise.