autoresearch

autoresearch is a skill for Claude Code, Codex from aleksbuss/orchestra. It costs 21 tokens per session (903 once invoked), scanned A, original, MIT.

An autonomous research workflow for training Nanochat, a small language-model training setup, on Apple Silicon computers. It searches for model designs and training settings that improve validation bits per byte, a measure of how well the model predicts text.

In plain words
What is it for?
Use it to modify and test the model architecture and training settings in Nanochat while leaving the data-preparation file unchanged and preserving memory for macOS.
Why use it?
It provides a defined experiment loop and hardware limits for improving the model without exhausting the computer's shared memory.

Skill for Claude CodeCodex

Written for no agent in particular: nothing here depends on one.

Good fit Use it to modify and test the model architecture and training settings in Nanochat while leaving the data-preparation file unchanged and preserving memory for macOS.

Compare 6 skills from other repositories ↓
Install with agentmods
npx agentmods add skills/aleksbuss/orchestra/autoresearch
Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

Any agent
npx skills add aleksbuss/orchestra --skill autoresearch
Clone the repo
git clone --depth 1 https://github.com/aleksbuss/orchestra

Made for: Claude Code, Codex.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for autoresearch

README.md
[![agentmods](https://agentmods.dev/badge/skills/aleksbuss/orchestra/autoresearch/github.svg)](https://agentmods.dev/skills/aleksbuss/orchestra/autoresearch)
Your own site
<a href="https://agentmods.dev/skills/aleksbuss/orchestra/autoresearch"><img src="https://agentmods.dev/badge/skills/aleksbuss/orchestra/autoresearch/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

agentmods 80×15 button for autoresearch

Your own site · 80×15
<a href="https://agentmods.dev/skills/aleksbuss/orchestra/autoresearch"><img src="https://agentmods.dev/badge/skills/aleksbuss/orchestra/autoresearch.svg" alt="Reviewed on agentmods" width="80" height="20"></a>
Per session 21 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 903 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00021 $0.00903
Opus 5 $0.00010 $0.00451
Sonnet 5 $0.00004 $0.00181
Haiku 4.5 $0.00002 $0.00090

Measured 9d ago against content hash 7dd8bf55a606, method: parsed. Prices are Anthropic first-party input rates as of 2026-09-09, from the pricing page.

Security

Grade A, and why

autoresearch scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 9d ago.

The scan reads SKILL.md. This mod also ships 1 executable file (setup-macos.sh), listed below but not scanned — reading those needs a real analyzer, not pattern matching.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

bundled-skills/autoresearch/SKILL.md · 63 lines

How it starts

The opening of the file, as written. The whole thing — 63 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Auto Research Autonomous Skill

Goal

You are an autonomous AI Deep Learning Researcher. Your overarching goal is to discover novel neural network architectures and hyperparameter configurations that achieve the lowest possible val_bpb (Validation Bits Per Byte) on the nanochat training setup.

The Environment

You have been provided with an experimental sandbox optimized for Apple Silicon (MLX). Note: A hardware profiler has already run and scaled the TOTAL_BATCH_SIZE in train.py to match the host's exact Unified Memory limit. Do not decrease this batch size, as it is already optimal. You may focus on architecture changes.

The directory contains three main files:

  1. prepare.py — constants, data prep, and resource management utilities. DO NOT MODIFY THIS FILE.
  2. train.py — the model architecture, optimizer, and training loop. THIS IS THE ONLY FILE YOU WILL MODIFY.
  3. setup-macos.sh — an initialization script.

Hardware Constraints & Safety

CRITICAL: You are running on a local machine with limited Unified Memory (RAM).

  • You MUST leave at least 20% of RAM free for the Operating System to prevent system hangs.
  • Before proposing an architecture, mentally estimate its size.
  • If you see a CRITICAL RESOURCE ERROR in the logs, it means your proposed model is too large. You MUST revert and try a smaller configuration (e.g., lower DEPTH or n_embd).
  • On a 16GB machine, DEPTH=12 is likely too high. Aim for DEPTH=4 to DEPTH=8.

The Loop Rules

You must operate in a strict, infinitely repeating loop. For each iteration, perform the following steps:

Step 1: Initialization (Only if first run)

If this is the very first time you are running, you must execute bash setup-macos.sh using the code_execution tool. This will download the MLX fork, install uv, and download the TinyStories dataset. Do not proceed until this completes successfully.

Step 2: Ideation

Propose a single, specific modification to train.py. This could be:

  • Changing a hyperparameter (e.g., TOTAL_BATCH_SIZE, learning rate, DEPTH).
  • Modifying the architecture (e.g., changing the attention mechanism, adding normalization layers, swapping the optimizer).
  • State your hypothesis clearly in the chat: why do you think this change will improve the model?

Read the full file on GitHub · 63 lines

Files

What ships with it

2 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 9d ago First seen · 63 lines · 21 tokens per session scan A 7dd8bf55a606

Subscribe to this mod's changes

autoresearch is a skill published in the GitHub repository aleksbuss/orchestra (2 stars, last pushed yesterday), licensed MIT. It adds 21 tokens to every session and 903 once invoked, about $0.0001 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.