data-analysis

Data Analysis Pipeline

Generate, execute, and verify analysis scripts across R, Python, Stata, and Julia.

Modes

Mode	What it does	Phases
EDA	Exploratory data analysis only	1–2
Estimation	Estimation + publication output (requires locked design)	1, 3–4
Full	Complete pipeline	1–5

Default: Full. Detect mode from user request or ask if ambiguous.

When to Use

• "Analyse this data" / "Run EDA on this CSV" / "Estimate the model"
• "Generate results tables" / "Create publication figures"
• Any task requiring data → script → output pipeline

When NOT to Use

• Experimental design or power analysis → experiment-design
• Generating synthetic data for testing → synthetic-data
• Auditing identification strategy → causal-design
• Proofreading or compiling the paper → proofread, latex

Shared References

• Method probing questions: shared/method-probing-questions.md — ask before running any analysis
• Validation tiers: shared/validation-tiers.md — declare tier before examining results
• Escalation protocol: shared/escalation-protocol.md — escalate when methodology answers are vague
• Distribution diagnostics: shared/distribution-diagnostics.md — mandatory DV checks before model selection
• Engagement-stratified sampling: shared/engagement-stratified-sampling.md — stratify by engagement tiers for social media data
• Inter-coder reliability: shared/intercoder-reliability.md — per-category reliability for content analysis and LLM annotation

Workflow

Phase 1: Setup

1. Detect project structure: Read CLAUDE.md, check for data/, code/, paper/ directories.
2. Detect language: Check existing scripts, user preference, or ask. Read shared/multi-language-conventions.md for the chosen language's conventions.
3. Locate data: Find datasets in data/raw/ or data/processed/. Never modify data/raw/ (per data-sensitivity rule). For social-media datasets, follow shared/engagement-stratified-sampling.md when constructing analysis samples.
4. Confirm validation tier per shared/validation-tiers.md. Tier dictates claim-strength language allowed in Phase 4 outputs and how strict the locked-design gate (step 5) is enforced.
5. Check for locked design: Look for analysis plan in log/plans/, .context/project-recap.md, or MEMORY.md estimand registry. If running Estimation or Full mode and no design exists, stop and warn: "No locked research design found. Run experiment-design or causal-design first, or confirm the specification before proceeding." Use shared/method-probing-questions.md to probe gaps if the user pushes back on the gate.