small-data-ml

Small-Data Machine Learning for Materials and Chemistry

Description

This skill covers machine learning on small scientific datasets (roughly 20–2000 samples) in materials science and chemistry: leakage-safe splitting strategies for correlated structures and compositions, baseline model selection, descriptor screening, uncertainty quantification, learning curves, model selection under limited data, transfer learning from pretrained representations, and rigorous reporting of performance including confidence intervals and negative results. Invoke this skill when building a property predictor, surrogate model, or screening model from a dataset small enough that every methodological choice — splitting strategy, baseline, hyperparameter tuning procedure — materially affects the reported metrics.

Domain Context

Most published materials ML models are trained on fewer than a few thousand samples. This is not a weakness — DFT, synthesis, and characterization are expensive — but it creates methodological traps that are easy to miss and hard to detect from the outside. High R² on a random 80/20 split of a 200-sample dataset is not evidence of a useful model; it may reflect correlation leakage, a narrow target range, or an over-tuned hyperparameter.

Why scientific small-data is different from standard ML small-data:

In general ML, small-data problems usually assume i.i.d. samples. Materials datasets violate this in structured ways:

• Compositional correlation: Two alloys that differ by 1 at% are nearly identical in descriptor space and probably in target space. Random splitting separates near-duplicates across train and test, making the model look more accurate than it generalizes.
• Structural prototype correlation: All rocksalt oxides share the same topology; a model trained on MgO, NiO, CoO will trivially interpolate to FeO even under random splitting.
• Database-sourced duplicates: Materials Project, AFLOW, and OQMD share many structures computed by different methods; deduplication before splitting is mandatory.
• Experimental campaign structure: Samples in a synthesis campaign often share substrate, precursor, or equipment — a group that must be held out together, not split randomly.
• Active-learning iteration structure: Samples from later iterations are in regions near those from earlier iterations. Splitting across iterations pretends the model was trained on future knowledge.