autoresearch

Autoresearch

Autonomous LLM optimization: the agent iterates on train.py, runs 5-minute GPU experiments, measures validation bits-per-byte (val_bpb), and publishes the best checkpoint as a sellable Ollama model.

When to Use

• Optimizing a base model for a specific domain or task
• Running automated training experiments to improve val_bpb
• Publishing an optimized model checkpoint to Ollama
• Selling an optimized model via x402 payment-gated inference

When NOT to Use

• Selling an existing model without optimization — use sell
• Buying remote inference — use buy-x402
• Cluster diagnostics — use obol-stack

Quick Start

1. Prepare Data

Place your training and validation data in the autoresearch working directory:

autoresearch/
  train.bin        # training data (tokenized)
  val.bin          # validation data (tokenized)
  train.py         # training script (agent modifies this)
  results.tsv      # experiment log (appended by each run)

2. Run Experiments

The agent modifies train.py and runs experiments in a loop. Each experiment:

• Has a 5-minute time budget on GPU
• Produces a checkpoint and a val_bpb measurement
• Is tracked as a git commit with status (keep/discard) in results.tsv

The results.tsv file is tab-separated with columns:

commit_hash	val_bpb	status	description
a1b2c3d	1.042	keep	baseline transformer
e4f5g6h	1.038	keep	added RMSNorm
i7j8k9l	1.051	discard	unstable lr schedule

3. Publish the Best Model

Once experiments are complete, use publish.py to find the best checkpoint, register it with Ollama, and optionally sell it:

# Publish to Ollama only
python3 scripts/publish.py /path/to/autoresearch

# Publish and sell via x402
python3 scripts/publish.py /path/to/autoresearch \
  --sell \
  --wallet 0xYourWalletAddress \
  --price 0.002 \
  --chain base-sepolia

Commands

Command	Description
publish.py <dir>	Find best experiment, create Ollama model, generate provenance
publish.py <dir> --sell --wallet <addr> --price <p> --chain <c>	Publish and sell via obol sell inference