meta-ml-screener

Meta ML Screener

Use this skill when machine learning will assist a systematic review or meta-analysis.

Core Principle

ML can reduce workload, prioritize records, extract candidates, and explore heterogeneity. It should not hide eligibility criteria, final inclusion decisions, or effect-size verification.

Intake

Identify the ML task:

• deduplication;
• title/abstract screening;
• full-text triage;
• study design classification;
• outcome classification;
• extraction assistance;
• risk-of-bias triage;
• topic modeling;
• moderator discovery.

Identify:

• labeled data available;
• human verification plan;
• recall requirement;
• audit log format;
• software or platform;
• whether LLMs are used.

Load references/ml-assisted-review.md for task-specific guidance.

Workflow

1. Define ML role and what decisions remain human.
2. Create seed labels or validation set.
3. Define features, model, or prompt schema.
4. Run prioritization/classification/extraction.
5. Record scores, labels, and decisions.
6. Validate recall or extraction accuracy.
7. Escalate uncertain records to human review.
8. Export decisions, model scores, prompts/schemas, and human adjudication.
9. Report ML use transparently.

Use:

• templates/screening-log.md for a human-readable log.
• templates/screening-log-schema.csv for machine-readable logging.
• templates/example-screening-log.csv for a minimal example.
• scripts/validate_screening_log.py to check required fields, exclusion reasons, duplicate record IDs, and human follow-up decisions.

Output Modes

ML Screening Plan

ML task:
Human decision point:
Training/seed labels:
Validation metric:
Audit log:
Stopping rule:
Failure modes:
Reporting sentence:

Extraction Schema

Field:
Definition:
Source anchor:
Confidence:
Human verification:

Guardrails

• Do not exclude records solely because the model is confident unless the protocol explicitly allows it and recall is validated.
• Do not use LLM-extracted numbers without source anchors and verification.
• Do not treat ML-discovered moderators as confirmatory.
• Do not hide prompt/model/version details if they affect review decisions.
• Do not let automation erase exclusion reasons or reviewer accountability.
• Do not accept a screening log as auditable unless human decisions and exclusion reasons are recorded.