sklearn-explainability

scikit-learn - Explainability & Interpretability

In scientific research, a model's "why" is as important as its "what". This guide focuses on tools that reveal the decision-making process of machine learning models, ensuring they are scientifically valid and not just overfitting on artifacts.

When to Use

• Validating that a model uses physically meaningful features (e.g., in drug discovery).
• Identifying biases or "shortcuts" the model has learned from the training data.
• Explaining individual predictions to non-experts (Local explanations).
• Ranking the global impact of variables on a complex system (Global explanations).
• Scientific auditing and regulatory compliance.

Core Principles

1. Model-Specific vs. Model-Agnostic

• Model-Specific: Tools like feature_importances_ in Random Forests. Fast but tied to one architecture.
• Model-Agnostic: Tools like SHAP or Permutation Importance. Work on any model (SVM, MLP, etc.) but are more compute-intensive.

2. Global vs. Local Explanations

• Global: How does the feature "Temperature" affect the model overall?
• Local: Why did the model predict "Reaction Failed" for this specific sample?

3. Feature Importance vs. Feature Contribution

Importance tells you if a feature is used; Contribution tells you how it changed the output (positive or negative).

Quick Reference: Built-in Inspection

from sklearn.inspection import permutation_importance, PartialDependenceDisplay

# 1. Permutation Importance (Better than default tree importance)
result = permutation_importance(model, X_test, y_test, n_repeats=10)
print(result.importances_mean)

# 2. Partial Dependence Plots (How one feature affects prediction)
PartialDependenceDisplay.from_estimator(model, X, features=['temp', 'pressure'])

Critical Rules

✅ DO

• Prefer Permutation Importance over default RandomForest.feature_importances_ - Default importance is biased toward high-cardinality features (like unique IDs).
• Use PartialDependenceDisplay - To visualize the relationship between a feature and the target (Linear, Exponential, or Sigmoid).
• Scale Features before Interpretability - Many models (like Logistic Regression) require scaling for their coefficients (β) to be comparable.
• Check Feature Correlations - If two features are highly correlated, importance will be split between them, making both look "less important" than they are.