mlops-industrialization

MLOps Industrialization

Goal

To convert experimental code (notebooks/scripts) into a high-quality, distributable Python package. This skill enforces the src/ layout, a Hybrid Paradigm (OOP structure + Functional purity), and Strict Configuration to ensure scalability, security, and maintainability.

Prerequisites

• Language: Python 3.14
• Manager: uv
• Context: Moving from notebooks/ to src/.

Instructions

Preserve existing work and repository visibility. Prepare local changes first; commit, push, change remote settings, publish, or deploy only with user authorization.

1. Packaging Structure (src Layout)

Adopt the src layout to prevent import errors and separate source from tooling.

1. Directory Tree:

   my-project/
   ├── pyproject.toml       # Dependencies & Metadata
   ├── uv.lock              # Pinned Python dependencies
   ├── mise.toml            # Task vocabulary & pinned tools
   ├── mise.lock            # Pinned tool binaries
   ├── AGENTS.md            # Instructions for AI agents
   ├── README.md
   └── src/
       └── my_package/      # Main package directory
           ├── __init__.py
           ├── io/          # Side-effects (Datasets, APIs)
           ├── domain/      # Pure business logic (Models, Features)
           └── application/ # Orchestration (Training loops, Inference)
   

2. Configuration: Use pyproject.toml for all build metadata and dependencies.

2. Modularity & Paradigm (Hybrid Style)

Balance structure with predictability.

1. Domain Layer (Pure):
   • Rule: Code here must be deterministic and free of side effects (no I/O).
   • Use Case: Feature transformations, Model architecture definitions.
   • Style: Functional (pure functions) or Immutable Objects (dataclasses).
2. I/O Layer (Impure):
   • Rule: Isolate external interactions here.
   • Use Case: Loading data from S3, saving models to disk, logging to MLflow.
   • Style: OOP (Classes to manage connections/state).
3. Application Layer (Orchestration):
   • Rule: Wire Domain and I/O together.
   • Use Case: Tuning, Training, Inference, Evaluation, etc.