claude-scaffold: Skill for Claude Code

ML Data Handling

When to Load This Skill

Load when working with: pickle, ONNX, Parquet, Feather, HDF5, large datasets, S3/Object Storage, DVC-like versioning, model artifacts, data pipelines.

Core Principle

Binary ML artifacts (weights, embeddings, datasets) are NEVER committed to Git. They live in object storage (Yandex Cloud Object Storage — S3-compatible) or are reproducible via pipeline. Paths and versions are tracked in code; actual data is not.

Directory Convention

project-name/
├── data/
│   ├── raw/          # gitignored — original client data, immutable
│   ├── interim/      # gitignored — intermediate transformations
│   ├── processed/    # gitignored — final features ready for training
│   └── .gitkeep      # committed — preserves structure
├── models/
│   ├── weights/      # gitignored — .pt, .onnx, .safetensors
│   └── .gitkeep
└── artifacts/        # gitignored — experiment outputs

Data Versioning Strategy

We do not use DVC (adds friction). Instead: manifest files committed to Git.

Each data version has a corresponding data/manifest.json:

{
  "version": "1.2.0",
  "created_at": "2026-03-01T10:00:00Z",
  "splits": {
    "train": {
      "path": "s3://bucket/datasets/project/v1.2.0/train.parquet",
      "rows": 45000,
      "sha256": "a3f2..."
    },
    "val": {
      "path": "s3://bucket/datasets/project/v1.2.0/val.parquet",
      "rows": 5000,
      "sha256": "b7c1..."
    }
  },
  "preprocessing": {
    "script": "scripts/preprocess.py",
    "commit": "abc123"
  }
}

S3 / Yandex Cloud Object Storage Adapter

import boto3
from botocore.config import Config

from src.project_name.core.config import settings


def get_s3_client():
    return boto3.client(
        "s3",
        endpoint_url="https://storage.yandexcloud.net",
        aws_access_key_id=settings.yc_access_key_id,
        aws_secret_access_key=settings.yc_secret_access_key,
        config=Config(signature_version="s3v4"),
        region_name="ru-central1",
    )


async def download_artifact(s3_key: str, local_path: str) -> None:
    client = get_s3_client()
    client.download_file(settings.yc_bucket_name, s3_key, local_path)


async def upload_artifact(local_path: str, s3_key: str) -> None:
    client = get_s3_client()
    client.upload_file(local_path, settings.yc_bucket_name, s3_key)