nagi-bench: Instructions file for Codex

AGENTS.md

Guidance for humans and coding agents contributing to NAGI BENCH data (https://bench.nagi.fun/). Read this before opening a PR.

What this repo is

The public data repo: community-contributed one-shot LLM eval artifacts. You add a model's raw output for a case; the site renders it verbatim. This repo holds data only — no site code, no build of the page you see.

A contribution is exactly two files

1. The artifact — the raw model output, dropped at outputs/<artifactDir>/<case-id>.<ext> (e.g. a single-file .html or .svg).
2. The registration — models/<model-id>.json declaring it.

Do not touch anything else. In particular:

• Do not edit README.md / README.en.md. The Registry tables are generated from models/*.json and synced automatically after merge. Hand edits are overwritten.
• Do not edit cases.json. Cases are maintainer-owned.

models/<model-id>.json schema

{
  "label": "DeepSeek V4 Flash",   // human-facing model name (required)
  "vendor": "DeepSeek",            // who makes the model (required)
  "harness": "Reasonix",           // the tool/agent/UI you ran it in (required)
  "effort": "Max",                 // thinking budget / effort level (required)
  "artifactDir": "deepseek-v4-flash/reasonix-max", // optional; defaults to <model-id>
  "order": 50,                      // optional; newer versions sort first automatically (scripts/model-order.ts) — order only breaks ties within the same version
  "runs": {
    "skeleton-watch": {
      "note": {                     // REQUIRED, bilingual, zh first, no emojis
        "zh": "在 Reasonix 中以 DeepSeek V4 Flash（Max）一次生成，未修改。",
        "en": "One-shot by DeepSeek V4 Flash (Max effort) in Reasonix, unmodified."
      },
      "contributor": "your-github-username"
    }
  }
}

Naming rules

• The filename IS the entry id — lowercase letters, digits and dashes only (deepseek-v4-flash-reasonix). It is the stable voting key and the default outputs/ folder, so name it after the model, not the harness. Bad: reasonix-reasonix-max. Good: deepseek-v4-flash-reasonix.
• artifactDir may be one or two dash-case segments (deepseek-v4-flash/reasonix-max); set it when several entries share a model folder. Leave it out to default to the id.