modeling-strategy-guide

Modeling Strategy Guide

A skill for strategic statistical modeling applied to academic research. Covers advanced modeling decisions, experimental design, causal inference, feature engineering, and the critical thinking required to move from data to defensible conclusions.

Overview

Senior data scientists distinguish themselves not by knowing more algorithms but by asking better questions, designing cleaner experiments, and being honest about what the data can and cannot tell them. This skill translates that professional discipline into a research context, helping academics apply modern data science practices to their empirical work. It covers the strategic decisions that matter most: when to use simple models versus complex ones, how to establish causality rather than mere correlation, and how to communicate uncertainty honestly.

The skill is particularly useful for researchers working with observational data who need causal inference techniques, those designing randomized experiments who need proper power calculations and analysis plans, and anyone building predictive models who needs to avoid common overfitting and leakage pitfalls.

Strategic Modeling Decisions

Model Selection Philosophy

Decision Framework:
1. Start with the simplest model that could answer your question
2. Add complexity only when diagnostics reveal inadequacy
3. Prefer interpretable models unless prediction accuracy is the sole goal
4. Always have a baseline (mean, majority class, last observation)

Model Complexity Ladder:
  Level 1: Descriptive statistics, cross-tabulations
  Level 2: Linear/logistic regression
  Level 3: Regularized regression (Lasso, Ridge, Elastic Net)
  Level 4: Tree ensembles (Random Forest, Gradient Boosting)
  Level 5: Deep learning (only with sufficient data and clear justification)

Feature Engineering Principles

import pandas as pd
import numpy as np

def engineer_features(df: pd.DataFrame, config: dict) -> pd.DataFrame:
    """
    Apply systematic feature engineering based on domain knowledge.

    config example:
    {
        'log_transform': ['income', 'citations'],
        'interactions': [('experience', 'education')],
        'polynomial': {'age': 2},
        'time_features': 'date_column',
        'lag_features': {'metric': [1, 7, 30]}
    }
    """
    df = df.copy()

    # Log transforms for right-skewed variables
    for col in config.get('log_transform', []):
        df[f'{col}_log'] = np.log1p(df[col])

    # Interaction terms
    for col_a, col_b in config.get('interactions', []):
        df[f'{col_a}_x_{col_b}'] = df[col_a] * df[col_b]

    # Polynomial features
    for col, degree in config.get('polynomial', {}).items():
        for d in range(2, degree + 1):
            df[f'{col}_pow{d}'] = df[col] ** d

    # Time-based features
    if 'time_features' in config:
        time_col = config['time_features']
        df[time_col] = pd.to_datetime(df[time_col])
        df[f'{time_col}_month'] = df[time_col].dt.month
        df[f'{time_col}_dayofweek'] = df[time_col].dt.dayofweek
        df[f'{time_col}_quarter'] = df[time_col].dt.quarter

    return df