mlops-industrialization

MLOps Industrialization

Goal

To convert experimental code (notebooks/scripts) into a high-quality, distributable Python package. This skill enforces the src/ layout, a Hybrid Paradigm (OOP structure + Functional purity), and Strict Configuration to ensure scalability, security, and maintainability.

Prerequisites

• Language: Python 3.14
• Manager: uv
• Context: Moving from notebooks/ to src/.

Instructions

1. Packaging Structure (src Layout)

Adopt the src layout to prevent import errors and separate source from tooling.

1. Directory Tree:

   my-project/
   ├── pyproject.toml       # Dependencies & Metadata
   ├── uv.lock              # Pinned Python dependencies
   ├── mise.toml            # Task vocabulary & pinned tools
   ├── mise.lock            # Pinned tool binaries
   ├── AGENTS.md            # Instructions for AI agents
   ├── README.md
   └── src/
       └── my_package/      # Main package directory
           ├── __init__.py
           ├── io/          # Side-effects (Datasets, APIs)
           ├── domain/      # Pure business logic (Models, Features)
           └── application/ # Orchestration (Training loops, Inference)
   

2. Configuration: Use pyproject.toml for all build metadata and dependencies.

2. Modularity & Paradigm (Hybrid Style)

Balance structure with predictability.

1. Domain Layer (Pure):
   • Rule: Code here must be deterministic and free of side effects (no I/O).
   • Use Case: Feature transformations, Model architecture definitions.
   • Style: Functional (pure functions) or Immutable Objects (dataclasses).
2. I/O Layer (Impure):
   • Rule: Isolate external interactions here.
   • Use Case: Loading data from S3, saving models to disk, logging to MLflow.
   • Style: OOP (Classes to manage connections/state).
3. Application Layer (Orchestration):
   • Rule: Wire Domain and I/O together.
   • Use Case: Tuning, Training, Inference, Evaluation, etc.