batch-inference

Batch Inference — scheduled scoring jobs

Real-time /predict serves one request at a time. Batch scoring reuses the SAME feature engineering + model artifact but runs over millions of rows in one process. Sharing the code path prevents training/serving skew AND training/batch skew — the classic silent-ML-failure mode.

When NOT to use this skill

• Intermittent one-offs — script + argparse inside the service repo suffices; no need for a K8s CronJob.
• Scoring must happen inline (form submission, checkout flow) — that is real-time; use /predict.
• Training-adjacent batch (e.g., feature backfill for re-training) — that belongs in the training pipeline, not a batch scoring job.

Architecture

templates/service/
├── app/                 # real-time API (unchanged)
├── src/{service}/
│   ├── predictor.py     # EXISTING: shared predict() — used by both paths
│   └── batch.py         # NEW: batch runner — main() entry
└── k8s/base/
    └── cronjob-batch.yaml   # NEW: scheduled execution

Key principle: both paths import predictor.predict(). Never duplicate feature engineering. Any change to prediction logic auto- propagates to both.

Execution flow

Step 1 — Confirm intent (AUTO, 30s)

Agent confirms:

• service name + code path exists
• business cadence (hourly / daily / weekly / monthly)
• input source (BigQuery table, S3 parquet, GCS folder)
• output sink (parquet partitioned by date, BigQuery table, RDS)
• expected row count per run (sizes the K8s resources)

Step 2 — Scaffold src/{service}/batch.py (AUTO)

The scaffolded module MUST:

1. Read input via a PANDAS/Polars DataFrame
2. Validate via the SAME Pandera schema used in training (rule 08)
3. Call predictor.predict_batch(df) — same code as /predict
4. Emit predictions as partitioned parquet with prediction_id, entity_id, model_version, prediction_score, timestamp
5. Also call log_prediction() for each row — the closed-loop flywheel works for batch too (ADR-006, D-20/D-22)
6. Emit metrics:
   • {service}_batch_rows_processed_total
   • {service}_batch_duration_seconds
   • {service}_batch_errors_total