explainability-by-default

Explainability by Default

Every AI system you build must be able to explain its decisions. This is not an add-on — it is a design constraint. Build explanation capability first, then the model.

The Explainability Design Question

Before choosing a model architecture, answer this:

Who needs to understand this decision, and what do they need to know?

Audience	What They Need	Approach
Affected citizen	"Why was I declined/flagged/scored?"	Plain-English local explanation
Frontline staff	"What drove this score for this person?"	Feature importance for this case
Auditor / regulator	"How does this system work systematically?"	Global model behaviour + documentation
Developer / data scientist	"Is the model behaving as expected?"	SHAP values, attention maps, partial dependence

Design for all relevant audiences before writing model code.

---

Step 1: Choose an Interpretable-First Architecture

Prefer interpretable models unless you have a clear, documented reason not to:

Decision tree → Logistic regression → Linear SVM
     ↓ (only if accuracy genuinely requires it)
Random Forest + SHAP → Gradient Boosting + SHAP
     ↓ (only if task genuinely requires it)
Deep neural network + explanation layer
     ↓ (last resort — requires extra governance)
Black-box model (requires independent audit)

Rule: Use the simplest model that meets your accuracy requirements. Document why you didn't use a simpler one.

---

Step 2: Add Explanation Capability at Build Time

For Scikit-learn Models

import shap

# Train model
model.fit(X_train, y_train)

# Build explainer at training time — not as an afterthought
explainer = shap.TreeExplainer(model)  # for tree-based models
# or
explainer = shap.LinearExplainer(model, X_train)  # for linear models
# or
explainer = shap.KernelExplainer(model.predict, shap.sample(X_train, 100))  # for any model

# Save explainer alongside model
import joblib
joblib.dump({'model': model, 'explainer': explainer}, 'model_artifacts/model_with_explainer.pkl')