ML-MLOps-Portfolio: Skill for Claude Code

Concept Drift Analysis

This skill is the RCA counterpart to drift-detection. Where PSI detects feature-distribution change, this skill detects and diagnoses performance degradation using delayed ground-truth labels.

Inputs you need

• reports/performance.json from the most recent CronJob run (or manual run)
• Baseline metrics from training: models/baseline_metrics.json
• (Optional) drift_report.json to cross-reference with PSI

Decision tree

                    Performance alert fired
                             │
               ┌─────────────┴─────────────┐
         Global AUC low?             Only sliced AUC low?
               │                             │
       ┌───────┴───────┐           ┌─────────┴─────────┐
   Data drift         No drift   Drift in same slice?   No drift?
       │                │               │                 │
  Retrain on       Label noise     Targeted retrain   Label quality
  fresh data       or real         or feature fix     in that segment
                   concept drift

Step 1: Read the report

jq '.status, .global, .alerts' reports/performance.json
jq '.slices' reports/performance.json   # per-slice breakdown

Identify which slices fired alerts. Each entry has:

{ "slice_name": "by_country", "slice": "country=ES", "metric": "auc",
  "value": 0.58, "threshold": 0.65 }

Step 2: Distinguish global vs sliced

# Global AUC
jq '.global.auc' reports/performance.json
# All slice AUCs, sorted ascending
jq -r '.slices | to_entries[] | .value | to_entries[] | [.key, .value.auc] | @tsv' \
  reports/performance.json | sort -k2 -n

• Global degradation (many slices low) → population-wide concept drift
• Single-slice degradation (others healthy) → subpopulation issue
• Healthy global, healthy slices → false alarm from window size

Step 3: Cross-reference with data drift

Correlate slice degradation with feature PSI in the SAME window: