senior-data-scientist

Senior Data Scientist

Overview

Build end-to-end data science workflows from data exploration through model deployment. This skill covers data preprocessing, feature engineering, model selection, hyperparameter tuning, cross-validation, experiment tracking with MLflow/W&B, statistical testing, visualization with matplotlib/seaborn/plotly, and Jupyter notebook best practices.

Announce at start: "I'm using the senior-data-scientist skill for data science workflow."

---

Phase 1: Data Understanding

Goal: Profile the dataset and establish a baseline before any modeling.

Actions

1. Load and profile the dataset (shape, types, distributions)
2. Identify missing values, outliers, and data quality issues
3. Perform exploratory data analysis (EDA)
4. Define the target variable and success metrics
5. Establish baseline performance

Baseline Models (Always Start Here)

Task	Baseline Model	Why
Classification	Majority class classifier	Lower bound for accuracy
Classification	Logistic regression	Simple, interpretable
Regression	Mean predictor	Lower bound for RMSE
Regression	Linear regression	Simple, interpretable
Time series	Naive forecast (previous value)	Lower bound for MAE
Time series	Seasonal naive	Captures basic seasonality

STOP — Do NOT proceed to Phase 2 until:

• Dataset is profiled (shape, types, distributions)
• Missing values and outliers are documented
• Target variable is defined
• Success metrics are chosen
• Baseline performance is established

---

Phase 2: Feature Engineering

Goal: Transform raw data into features that improve model performance.

Actions

1. Handle missing values (imputation strategy)
2. Encode categorical variables
3. Scale/normalize numerical features
4. Create derived features
5. Feature selection (remove redundant/irrelevant)

Missing Value Strategy Decision Table

Strategy	When to Use	Implementation
Drop rows	< 5% missing, MCAR	df.dropna()
Mean/Median	Numerical, no outliers	SimpleImputer(strategy='median')
Mode	Categorical	SimpleImputer(strategy='most_frequent')
KNN Imputer	Structured missing patterns	KNNImputer(n_neighbors=5)
Iterative	Complex relationships	IterativeImputer()
Flag + Impute	Missingness is informative	Add is_missing column + impute