feature-engineering

Feature Engineering

Build feature pipelines that transform raw data into model-ready inputs using scikit-learn.

When to Use

• User needs to encode categorical variables (one-hot, ordinal, label)
• User needs to scale or normalize numeric features
• User wants to select the best features for a model
• User needs to create derived features (interactions, aggregations, date parts)
• User needs to remove outliers from a dataset

Available Operations

1. Clean data and remove outliers (IQR method)
2. Encode categorical features (OneHot, Ordinal, Label)
3. Scale numeric features (Standard, MinMax, Robust)
4. Create derived features (date parts, interactions, aggregations)
5. Select top features (statistical tests, model importance)
6. Build a reusable sklearn ColumnTransformer pipeline

Multi-Step Workflow

Step 1: Install Dependencies

pip install pandas numpy scikit-learn joblib

Step 2: Load and Split Data

import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split

df = pd.read_parquet("data/processed/dataset_clean.parquet")

# Separate target
X = df.drop(columns=["target"])
y = df["target"]

# Split BEFORE any fitting -- prevents data leakage
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
print(f"Train: {X_train.shape}, Test: {X_test.shape}")

Step 3: Remove Outliers (IQR Method)

def remove_outliers_iqr(df: pd.DataFrame, columns: list[str]) -> pd.DataFrame:
    df = df.copy()
    for col in columns:
        Q1, Q3 = df[col].quantile([0.25, 0.75])
        IQR = Q3 - Q1
        mask = (df[col] >= Q1 - 1.5 * IQR) & (df[col] <= Q3 + 1.5 * IQR)
        before = len(df)
        df = df[mask]
        print(f"  {col}: removed {before - len(df)} outliers")
    return df

numeric_cols = X_train.select_dtypes(include=[np.number]).columns.tolist()
X_train = remove_outliers_iqr(X_train, numeric_cols)
y_train = y_train.loc[X_train.index]

Step 4: Build Encoding and Scaling Pipeline

from sklearn.preprocessing import StandardScaler, OneHotEncoder, OrdinalEncoder
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline

numeric_features = ["age", "salary", "experience"]
categorical_features = ["department", "city"]
ordinal_features = ["level"]

preprocessor = ColumnTransformer(
    transformers=[
        ("num", StandardScaler(), numeric_features),
        ("cat", OneHotEncoder(sparse_output=False, handle_unknown="ignore"), categorical_features),
        ("ord", OrdinalEncoder(categories=[["junior", "mid", "senior"]]), ordinal_features),
    ],
    remainder="drop"
)

# Fit on train only, transform both
X_train_transformed = preprocessor.fit_transform(X_train)
X_test_transformed = preprocessor.transform(X_test)
print(f"Features after transform: {X_train_transformed.shape[1]}")