scikit-learn-best-practices

Scikit-learn Best Practices

Expert guidelines for scikit-learn development, focusing on machine learning workflows, model development, evaluation, and best practices.

Code Style and Structure

• Write concise, technical responses with accurate Python examples
• Prioritize reproducibility in machine learning workflows
• Use functional programming for data pipelines
• Use object-oriented programming for custom estimators
• Prefer vectorized operations over explicit loops
• Follow PEP 8 style guidelines

Machine Learning Workflow

Data Preparation

• Always split data before any preprocessing: train/validation/test
• Use train_test_split() with random_state for reproducibility
• Stratify splits for imbalanced classification: stratify=y
• Keep test set completely separate until final evaluation

Feature Engineering

• Scale features appropriately for distance-based algorithms
• Use StandardScaler for normally distributed features
• Use MinMaxScaler for bounded features
• Use RobustScaler for data with outliers
• Encode categorical variables: OneHotEncoder, OrdinalEncoder, LabelEncoder
• Handle missing values: SimpleImputer, KNNImputer

Pipelines

• Always use Pipeline to chain preprocessing and modeling
• Prevents data leakage by fitting transformers only on training data
• Makes code cleaner and more reproducible
• Enables easy deployment and serialization

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import RandomForestClassifier

pipeline = Pipeline([
    ('scaler', StandardScaler()),
    ('classifier', RandomForestClassifier(random_state=42))
])

Column Transformers

• Use ColumnTransformer for different preprocessing per feature type
• Combine numeric and categorical preprocessing in single pipeline

Model Selection and Tuning

Cross-Validation

• Use cross-validation for reliable performance estimates
• cross_val_score() for quick evaluation
• cross_validate() for multiple metrics
• Use appropriate CV strategy:
  • KFold for regression
  • StratifiedKFold for classification
  • TimeSeriesSplit for temporal data
  • GroupKFold for grouped data