evidence-chain-builder

Evidence Chain Builder

Overview

scripts/evidence_chain.py splits a claim from its evidence and scores how well the evidence carries it. Every evidence item is tagged with a source type, and the type decides whether that item counts:

type	counts as independently verifiable
official — a primary document, filing, standard, or the issuing body itself	yes
paper — peer-reviewed literature, or a dataset with a stated method	yes
data — raw measurements, logs, exports the user can re-run	yes
internal — the user's own notes, records, or an undocumented internal number	no
assertion — the claim restated, someone's opinion, or no source stated	no

The score is the share of items that count, as a percentage. internal and assertion are not "worth less" by taste — they are items nobody outside the conversation can check, and that is the property the score is measuring.

Full taxonomy, edge cases and tagging rules: references/evidence-types.md.

The one boundary that matters

This tool does not decide whether a claim is true. It grades the evidence, and nothing else.

That boundary is the whole point. A grader that also pronounced on truth would reproduce exactly the failure it exists to catch — a confident verdict unsupported by anything. So the tool reports what can be checked and what cannot, and stops.

Two consequences to carry into your answer:

• A score of 100 does not mean the claim is correct. It means every item offered is the kind of thing that could be checked. Nobody has checked it yet.
• A low score does not mean the claim is false. It means the claim is not established, which is a different and more useful statement.

Never upgrade "not established" into "false", and never upgrade a high score into "true". Both are the same mistake in opposite directions.

Workflow

1. Separate the claim from the evidence before running anything. Write the claim as one sentence. If the user gave you three conclusions, that is three runs — a single chain will average unrelated things and hide the weak one.
2. Tag every item, and tag honestly. An item is official only if it names a document or issuing body that exists independently of the conversation. "Our records show" is internal. "Studies suggest" with no study is assertion. If you cannot tell what an item is, it is assertion until the user says otherwise — guessing a stronger type defeats the tool.
3. Run the tool (command below).
4. Read the verdict band, not just the number. The bands are defined below.
5. Name the gap concretely. Do not report "the evidence is weak". Report which items are unverifiable and what specifically would replace each one — "the failure-rate figure needs the QA report it came from, not a description of it".
6. Report the score and the gap together. A score without the unverifiable list invites the reader to treat the number as a quality grade, which it is not.