ai-ops

AI Ops

Overview

Deploying a machine learning model is the easy part. Keeping it correct, fast, fair, and cost-effective in production is where most AI projects fail. AI Ops (MLOps) bridges the gap between data science experimentation and production engineering — covering model deployment, monitoring, retraining, and governance.

The AI Ops contract: A model in production is software that happens to be probabilistic. It needs all the same operational rigor as any other service — plus additional checks for correctness drift, data distribution shifts, and fairness degradation.

When to Use

• Deploying a model to production for inference
• Setting up model monitoring and drift detection
• Designing retraining pipelines and strategies
• Building feature stores and serving infrastructure
• Implementing A/B testing for model variants
• Ensuring model governance, explainability, and compliance
• Optimizing inference latency and throughput

NOT for:

• Model training and experimentation (this is data science work)
• Building the initial model prototype
• Pure research without production intent

The AI Ops Process

Step 1: Define Model Success Criteria

Before deploying, define what "working" means:

MODEL SUCCESS CONTRACT:
├── Accuracy: [metric and threshold, e.g. AUC-ROC > 0.85]
├── Latency: [p50 < 50ms, p99 < 200ms]
├── Throughput: [min X requests/second]
├── Fairness: [demographic parity difference < 0.05]
├── Explainability: [SHAP/LIME available for all predictions]
├── Cost: [inference cost per request < $0.001]
└── Drift: [input distribution PSI < 0.2, concept drift < 5% accuracy drop]

Step 2: Design the Serving Architecture

SERVING PATTERN SELECTION:

Latency Requirement    → Architecture
─────────────────────────────────────────────────
< 10ms (real-time)     → In-process inference (embedded model)
10-100ms (near-real)   → Dedicated inference service (containerized)
100ms-1s (interactive) → Auto-scaled inference cluster
> 1s (batch)           → Async queue + worker pool (batch inference)