discovery-toolbox

DISCOVERY TOOLBOX

Load on demand, never wholesale. Route to the smallest relevant section. Usually activate 1–3 operators, execute them, then return to the research loop.

The toolbox is a repertoire, not a checklist.

---

A · VISUAL REASONING

Trigger: shape, heterogeneity, ranking, trajectories, subsets, or individual observations may matter.

• RAW + EXTREMES — inspect representative raw units, argmax/argmin, and matched non-hits.
• RANK / SHAPE VIEW — inspect the full ordered profile or curve, not only thresholds or summary metrics.
• SMALL MULTIPLES / MULTISCALE — repeat the same view across subsets, clades, batches, quality strata, sources, time, and analysis scales.
• RESIDUAL / INFLUENCE VIEW — inspect residual structure and sensitivity to influential observations/groups.
• PREDICTED FIGURES — generate expected visual patterns under competing hypotheses before looking at decisive evidence.
• CALIBRATED VISUAL INFERENCE — important visual-only claim → declare the expected pattern, compare against matched nulls/line-ups, and calibrate the visual reader on pure nulls.
• DISPLAY AUDIT — if a conclusion depends on axes, aspect ratio, encoding, aggregation, or overplotting, compute it directly or re-render appropriately.

---

B · INSTRUMENT / DETECTABILITY / INFORMATION

Trigger: new screen, weak/null result, uncertain calibration, lossy representation, or multi-fidelity pipeline.

• PERMANENT CALIBRATION HARNESS — reusable screen → maintain structure-preserving null + spike-in/recovery + empirical decoys/negative controls.
• TARGET RELIABILITY — estimate target noise, cross-source agreement, test–retest/version agreement, and effective independent units.
• POINT → CURVE — replace a weak point estimate with signal vs n / noise / quality / aggregation / threshold / scale.
• EXISTENCE / INFORMATION BUDGET — estimate expected detectable hits and whether target information × independent units can support the search space; insufficient capacity → change instrument/design, not estimator.
• TOY TRUTH — reproduce a known answer on a toy slice and/or generative simulation before interpreting large-scale nulls.
• COMMON-CAUSE FIRST — before explaining individual hits, test screen-wide technical or biological causes.
• SIGNAL/NOISE AUTOPSY — assume signal exists; identify what makes it unreadable: target noise, heterogeneity, wrong unit, aggregation, alignment, measurement error, etc.
• DIFFUSE-SIGNAL SWITCH — if individual hits fail but the statistic distribution shifts, use whole-distribution/set-level modeling.
• INFORMATION-LOSS / RATE–DISTORTION AUDIT — vary compression, aggregation, or preprocessing and locate where target-relevant information disappears.
• DPI / PROVENANCE LEAKAGE CHECK — derived features cannot legitimately contain target information unavailable in their declared inputs; violations trigger leakage/provenance investigation.
• MULTI-FIDELITY RECALL — cheap filters must preserve top-k/rank behavior at the real operating threshold and in important subgroups; otherwise hedge pruning aggressiveness.
• UNDETERMINED ≠ NEGATIVE — pipeline failure, missingness, timeout, or non-convergence is a third outcome; track its structure.