Borrowing it
Nothing to install: this file belongs to vlasenkoalexey/tpu_performance_autoresearch_wiki. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.
curl -O https://raw.githubusercontent.com/vlasenkoalexey/tpu_performance_autoresearch_wiki/main/.claude/skills/create-experiment/SKILL.mdgit clone --depth 1 https://github.com/vlasenkoalexey/tpu_performance_autoresearch_wikiWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/vlasenkoalexey/tpu_performance_autoresearch_wiki/create-experiment)<a href="https://agentmods.dev/skills/vlasenkoalexey/tpu_performance_autoresearch_wiki/create-experiment"><img src="https://agentmods.dev/badge/skills/vlasenkoalexey/tpu_performance_autoresearch_wiki/create-experiment/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/vlasenkoalexey/tpu_performance_autoresearch_wiki/create-experiment"><img src="https://agentmods.dev/badge/skills/vlasenkoalexey/tpu_performance_autoresearch_wiki/create-experiment.svg" alt="Reviewed on agentmods" width="80" height="20"></a>- NVIDIA SkillSpector warn
SkillSpector: 2 findings, up to medium
These are SkillSpector’s own severities. On a checked sample its high-severity flags on skills were ~96% false positives — a documented command, a public API, a “never do X” rule — so we show them as a caution to read, not a verdict. Why →
- medium Excessive Agency · line 38 Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.Fix: Add human-in-the-loop confirmation for destructive, irreversible, or high-impact operations. Never auto-execute commands that modify files, send data, or alter system state.
- medium Excessive Agency · line 111 Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.Fix: Add human-in-the-loop confirmation for destructive, irreversible, or high-impact operations. Never auto-execute commands that modify files, send data, or alter system state.
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00090 | $0.02972 |
| Opus 5 | $0.00045 | $0.01486 |
| Sonnet 5 | $0.00018 | $0.00594 |
| Haiku 4.5 | $0.00009 | $0.00297 |
Grade A, and why
create-experiment scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 11d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 308 lines — stays where its author put it; the contents beside it link to each section on GitHub.
You are bootstrapping a new model family for the autoresearch optimization loop. Follow this sequence — ask questions, then create files. Do NOT silently write files without confirming with the user.
Step 1 — Gather model details via AskUserQuestion
Ask these questions in order (some may be combined into a single AskUserQuestion call with multiple questions):
Question 1: "Model folder slug?" (free-text input expected)
- This becomes the folder name under
wiki/experiments/. Convention: lowercase, underscores, no spaces. - Examples:
gemma4,gemma5,qwen3_8B,llama4_70B. - The full folder will be
wiki/experiments/<slug>_autoresearch_optimization/.
Question 2: "Model display name?" (free-text)
- Human-readable name for documentation. Used in the model-level program.md title.
- Examples: "Gemma 4", "Gemma 5", "Qwen3 8B", "Llama 4 70B".
Question 3: "Which lanes?" (multi-select)
- Options:
tpu (PyTorch eager on TPU),jax (JAX + Flax NNX),torchax (torchax bridge),maxtext (MaxText). User can also type a custom lane name. - The lane folders will be created under
wiki/experiments/<slug>_autoresearch_optimization/.
Question 4: "Model sizes?" (free-text, comma-separated)
- Examples:
"3B, 24B","1B, 8B, 70B","125M, 1.3B". - These populate the
sizes:field in model page frontmatter and the architecture description.
Question 5: "Target hardware variants?" (free-text, comma-separated)
- Examples:
"v5p-8, v5p-32, v6e-16","v6e-8, v6e-32". - These populate the
hardwares:field. Used to build the variant matrix on the model page.
Question 6: "Target sequence length for MFU optimization?" (number input)
- Default: 8192.
- Examples: 8192 (most modern LLMs), 4096, 16384.
Question 7 (optional): "Architecture invariants?" (free-text, multiline)
- Things that must NOT change without approval. Examples: "8 parallel tracks, 4 experts top-2 routing", "GLU activation, fused attention", etc.
- Skip if user has nothing specific yet — they can fill in later.
Question 8 (optional): "Trainer entry-point pattern?" (free-text)
- Default:
torchtitan.experiments.<lane>.<slug>.train_minimal(matches the existing convention). - User can override if their model uses a different entry pattern.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 11d ago First seen · 308 lines · 90 tokens per session scan A 482127a740e8
create-experiment is a skill published in the GitHub repository vlasenkoalexey/tpu_performance_autoresearch_wiki (55 stars, last pushed 7d ago), licensed MIT. It adds 90 tokens to every session and 2,972 once invoked, about $0.0005 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
ml_inference_optimization
This document describes the Leeroopedia MCP tools available during the with-KB benchmark run. It is a standalone reference and is not fed to the agents automatically.
leeroopedia-mcp
Use Leeroopedia MCP to fetch grounded ML/AI best practices, build and review ML plans, debug failures, verify code/math correctness, and expand KB citations via getpage.
ml-research-lab
Machine-learning research loop for dataset curation, fine-tuning, evaluation, inference deployment, experiment tracking, and model explainability. Use when working on ML experiments, training data, model benchmarks, RunPod/GPU runs, classifier quality, vLLM/GGUF serving, SHAP-style model explanations, or…
experiment-audit
Use this skill for scientific and ML-research reasoning work — evaluating experimental claims, auditing training runs or ablations, checking whether a statistical claim holds up, assessing reproducibility, reconciling contradictory results, reviewing a paper's methodology or results section, writing reviewer-style…
midmem-research-tracker
Keep the repo's RESEARCH.md ledger current from the research ingestions already in the MidMem store: for every unevaluated weekly research ingestion, evaluate each paper's feasibility and applicable improvement for MidMem, and record it as Paper (cite + link) · Finding (the empirical claim that matters) · Decision…
sematyp
SemaTyP combines two data sources into a knowledge graph for drug discovery / repositioning.