hypothesize-and-design

Hypothesize & design

Starts from the .research/sources/<id>/ that search-papers had CI fetch. This is a loop, not a pass: read, hypothesize, notice what the literature does not settle, go back to search-papers for it, and refine until the hypothesis rests on passages you have actually read.

1. Read the fetched papers (.research/sources/<id>/fulltext.txt, with the search row in metadata.json) with your own tools — a paper runs to 80k+ characters, so read in parts or search within the file. Keep, per paper the research will rest on, its source id and the passages that matter (page and verbatim quote): preregister_record takes exactly these (passages get ids p1, p2, … within a source), so note which passages the hypothesis answers (its gap) and which the design follows; record both as quoted_passage_ids. A quote is matched against the snapshot ignoring line breaks, end-of-line hyphenation and ligatures, so copy it as the extracted text has it; keep it to one passage on one page (pages are separated by form feeds and a quote cannot span two).
2. Fix the compute target: ask the user if it is not known, and record GPU and arch (x86_64/aarch64) — the design and later the dependency lockfile depend on it.
3. Map the method onto the upstream code, then enumerate the knobs when the design reuses an existing implementation. The map: for each step of the method as the paper states it, the upstream module.Class.method that carries it. A step the upstream cannot carry unmodified is yours to write, and the map says so: which upstream function was considered and why it cannot be used — a deviation from the method, declared in the design's summary and the hypothesis's notes. Use the upstream for every step it can carry; a map that leaves most steps to you goes to the user before any code is written. Then the knobs: from the repository at the commit you will pin, list the keys of its config files, the keyword defaults of the entry points you will call, its module constants, and the prompt or resource files it reads (by reading the code until the inventory tool lands). Classify each as varied (in params, on the grid), fixed away from the upstream default (with the passage stating the default and the reason), or kept at the default (with the reason); a knob you decide not to consider is an assumption, not an omission. Declare all of it in the record: the repository goes under its paper's repositories (url, commit, files naming the whole package, method_entry = the module.Class.method whose call runs the method), and the design carries repository_integration (repository_id, extension_points = upstream names your adapter subclasses, overrides or replaces, written as the defining module's module.Class — scigym.api.api.LLM, not the re-export scigym.api.LLM — arguments = one module.Class.method.arg per knob you set, fixed or on the grid alike: the argument of the function that consumes the value — the upstream's, or a src/ function's for an own knob (seed, workers, a threshold); each is a key of every run's params holding the value that reaches the argument unchanged). The run hook records what every src/ function and the upstream were called with, and the gate compares each run's params with it.
4. Author the hypothesis and the design against the guide get_prompts(step="hypothesis_and_design") returns: it states what a hypothesis, a claim (statement, rationale, criterion, predicted interval, cited passages), a design and a run must contain, and ends in the exact literature and hypotheses arguments preregister_record takes. retrieve_models / retrieve_datasets list curated candidates; search_huggingface_hub when they lack what the design needs. A gap the papers at hand cannot confirm is a reason to go back to search-papers, not to assume. Write the prose in the user's working language; metric names stay English identifiers. The design's summary states every choice that fixes the hypothesis space, what the method gets to see, how it selects and how it is scored — enough to re-implement the run without the code — and lists each departure from the paper's method as paper: … / here: … / why: …. Depart only where the paper's form cannot be run; declaring a departure does not excuse it.
5. Agree the design with the user before any code is written: run ids, params, claims with criteria and predictions, tables, and the departure list from step 4 as a table. A consultation is not a go-ahead to build. Then write the agreed design to .research/design.json — exactly the literature and hypotheses arguments preregister_record will take — and commit it.