acceptance-checks

Acceptance checks

Most specified work is judged mechanically: a script opens your files, reads named columns, compares numbers to tolerances and stops at the first violation. When a task tells you how it will be judged, it has handed you the test suite. Write it down, run it, and only then deliver.

The failure this prevents is not sloppiness. It is a correct analysis delivered with a column named sample where the specification said sample_id, an angle in degrees where radians were required, or a fraction that sums to 0.999 where exactly one was expected.

When the output is a report a person or a model will read and grade (a trace, a write-up with prescribed headings), there is no mechanical judge to anticipate. The whole check is that the prescribed headings and files exist and that the numbers in the prose match the tables and the answer file: five lines, run once at the end. Do not build the report from a generator script, write validators that count its code blocks, or scan it for placeholder words; none of that is graded, and the minutes go to the analysis the grader does read. For such a task the rest of this skill does not apply.

Build the check list from the specification

Read the instruction and every file it points to — schema, README, format description, grading notes, example output, validation script — and extract:

• Paths and files: exact absolute paths, file names, which directory, whether extra files are allowed there, and whether a real file is required (not a symlink or a directory).
• Structure: column names and their order, key sets, array shapes and dtypes, index columns, required rows or identifiers, sort order, header text, encoding, delimiter.
• Values: units, scale factors, sign conventions, coordinate frames, allowed vocabularies and code tables, ranges, and how missing values are expressed.
• Templates and example strings: when the format shows a string ("<a>; <b>; <c>", an example row, a bracket notation), the separator with its spaces, the case, the bracket style and the field order are the format. Build the string from the template and compare it byte for byte with the example; a;b is not a; b to an exact match.
• Number formatting: decimal places, significant figures, and whether trailing zeros must be kept — 0.250 and 0.25 are the same number and different strings, and a checker that counts significant figures or matches a notation pattern rejects the second. Format with an explicit format string rather than by printing a float.
• Numeric criteria: tolerances (absolute or relative), thresholds, metrics and their exact definitions, how ties are broken, whether every case must pass or a mean suffices.
• Self-consistency: statements of the form "the totals must equal", "the fractions sum to one", "the reported value must match the file", "the durations follow from the timestamps".
• Named methods: the tool, version, language, contrast direction, filter, reference build and feature universe the task prescribes. When the reference numbers come from a specific pipeline, the method is part of the contract: a valid alternative shifts scale and row membership and is scored as wrong. Run the named tool at the named version; where that is impossible, disclose the substitute and its known differences in the methods file rather than swapping silently.
• Judged text: when a reader or a rubric grades the written trace or report alone, the check list includes every prescribed heading verbatim, one numbered step with its code and its intermediate number for each clause of the question, and named entities where the question names them. What is not in the text did not happen.