leaderboard: Skill for Claude Code

Model classification research

Maintain classification records separately from benchmark results. Read the selected entries in src/data/<benchmark>.json, the existing registry in src/data/model-research/, and any requested scope before editing. Preserve system names, scores, ranks, evaluation dates, and result-source links.

Evidence and identity

• Research the exact evaluated model version. Distinguish a model checkpoint from its harness and from a hosted system. A similarly named later release does not establish an earlier result's parameters, price, or weights availability.
• Prefer provider model cards, papers, official checkpoint inventories, code/configurations, and official prices. Actually retrieve the source; a search snippet or inaccessible URL is not proof. Credible secondary research can support a labeled estimate when primary evidence is incomplete.
• For weights, inspect the actual repository/file inventory. A public GitHub harness, an empty Hugging Face repository, a promised release, or an open base model does not prove the evaluated fine-tune is available. Record a failed retrieval separately from evidence of non-release.
• Read runtime tool configuration as well as the main model description. Include required planner, executor, summarizer, embedding, and reranking models. Exclude training-only teachers and scoring judges from the evaluated runtime stack. Do not infer a provider from an OpenAI-compatible API.
• Preserve source URLs, a short explanation, confidence, and the actual check date. Explain conflicts such as a paper's architectural parameter count versus a checkpoint inventory including auxiliary tensors. Do not silently replace a historical evaluation with the latest contents of a mutable page.

Classification rules

Use the first supported basis:

1. Published total parameters, in billions: Tiny ≤4; Small >4–40; Medium >40–150; Large >150. Record active MoE parameters separately. Active count, non-embedding count, quantized file size, and number of parallel agents do not replace total parameters.
2. Published token-price proxy if total parameters are unknown: blend = (3 × inputUsdPerMillion + outputUsdPerMillion) / 4. Small <0.15; Medium 0.15–1 inclusive; Large >1. Use standard uncached USD prices per million tokens for the evaluated version. Exclude batch discounts, cache discounts, promotions unless unavoidable and disclosed, per-query charges, tool fees, subscriptions, and total benchmark run cost. Record the relevant context tier.
3. Best supported provisional estimate when neither is known. Assign one size, label it Estimated, explain the inference and uncertainty, and use low confidence for unidentified hosted stacks. Store a parameter range only when evidence supports that range; do not invent a count to fit a category. If a range crosses a boundary, explicitly explain the chosen provisional category.