deploy-ml-model-serving

Deploy ML Model Serving

See Extended Examples for complete configuration files and templates.

Deploy machine learning models to production with scalable serving infrastructure, monitoring, and A/B testing.

When to Use

• Deploying trained models to production for real-time inference
• Setting up REST or gRPC APIs for model predictions
• Implementing autoscaling for variable load patterns
• Running A/B tests between model versions
• Migrating from batch to real-time inference
• Building low-latency prediction services
• Managing multiple model versions in production

Inputs

• Required: Registered model in MLflow Model Registry or trained model artifact
• Required: Kubernetes cluster or container orchestration platform
• Required: Serving framework choice (MLflow, BentoML, Seldon Core, TorchServe)
• Optional: GPU resources for deep learning models
• Optional: Monitoring infrastructure (Prometheus, Grafana)
• Optional: Load balancer and ingress controller

Procedure

Step 1: Deploy with MLflow Models Serving

Use MLflow's built-in serving for quick deployment of scikit-learn, PyTorch, and TensorFlow models.

# Serve model locally for testing
mlflow models serve \
  --model-uri models:/customer-churn-classifier/Production \
  --port 5001 \
  --host 0.0.0.0

# Test endpoint
curl -X POST http://localhost:5001/invocations \
  -H 'Content-Type: application/json' \
  -d '{
    "dataframe_records": [
      {"feature1": 1.0, "feature2": 2.0, "feature3": 3.0}
    ]
  }'

Docker deployment:

# Dockerfile.mlflow-serving
FROM python:3.9-slim

# Install MLflow and dependencies
RUN pip install mlflow boto3 scikit-learn

# Set environment variables
ENV MLFLOW_TRACKING_URI=http://mlflow-server:5000
# ... (see EXAMPLES.md for complete implementation)

Docker Compose for local testing:

# docker-compose.mlflow-serving.yml
version: '3.8'

services:
  model-server:
    build:
      context: .
      dockerfile: Dockerfile.mlflow-serving
# ... (see EXAMPLES.md for complete implementation)