code-competition-artifact-pipeline

Code Competition Artifact Pipeline

Problem

Kaggle Code Competitions (no internet, notebook-only submission) often have community baselines that depend on external artifact datasets:

• Pre-trained model weights (pickle/onnx)
• Feature matrices (CSV/numpy)
• Custom library wheels (.whl)

Forking these baselines fails unless you identify and attach ALL required datasets. The error messages are often cryptic ("file not found", "module not found").

Solution

Step 1: Read the Notebook's find_artifacts() Function

Most community baselines have a find_artifacts() or similar function that searches /kaggle/input/ for specific directory structures:

def find_artifacts():
    candidates = [
        "/kaggle/input/datasets/author/artifact-name",
        "/kaggle/input/artifact-name",
    ]
    # ... searches for repo/, weights/, wheels/, data/train.csv

This reveals: exact dataset names, expected file structure, and required Python packages.

Step 2: Search for Artifact Datasets

kaggle datasets list --search "competition-name artifacts"
kaggle datasets list --search "author-name"

Look for high-download datasets (500+ downloads = canonical bundle).

Step 3: Attach ALL Required Datasets

In kernel-metadata.json:

{
  "dataset_sources": [
    "author/competition-artifacts",
    "author/library-wheel"
  ]
}

Common missing datasets (from real failures):

Competition	Missing Dataset	Error
ROGII	ravaghi/wellbore-geology-prediction-artifacts	"data/train.csv not found"
ROGII	phongnguyn23021656/koolbox-offline	"No module named 'koolbox'"
Biohub	thibautgoldsborough/cellmot-baseline-artifacts	"repo/ not found"
NeuroGolf	(embedded in notebook, no external needed)	—

Step 4: Pipeline-Trim for Crashing Components

If the notebook has multiple pipelines (A/B) and one crashes:

• Pipeline A succeeds but Pipeline B errors → kernel ERROR → no submission scored
• Fix: delete Pipeline B cells, keep only Pipeline A + final submission write
• Alternative: wrap Pipeline B in try/except