subtext-annotate

subtext annotate

Turn a chunk of text into a subtext document, then enrich it by asking the user clarifying questions — things in the document that would be unclear to a reader — and recording the answers as sentence-level metadata. The JSON schema, parsing rules, and a worked example live in references/format.md — read that file before producing any JSON. The final artifact is delivered in a columnar encoding described in references/columnar.md, read at Step 4.

Workflow

Step 1 — Normalize to CommonMark

Convert the input to clean CommonMark markdown before structuring it:

• Preserve existing markdown structure (headings, lists, tables, code fences) as-is.
• For unstructured text (emails, plain prose), infer light structure: a # title, ## headings for clear topic shifts, lists where the text enumerates. Do not invent structure that isn't implied by the content.
• Convert inline links and bracketed references to [[ref-id]] citation markers and collect each URL into the sources map (see format.md).
• Normalize curly quotes to straight quotes and non-breaking spaces to regular spaces.
• If display math ($$ … $$) appears, emit it as a code block with language latex. Math handling is otherwise out of scope.

Show the normalized markdown to the user only if the input required significant restructuring; otherwise proceed.

Step 2 — Build the document tree

Convert the normalized markdown into subtext JSON following the rules in references/format.md. Key points:

• First # heading becomes metadata.title; remaining headings become nested sections.
• Split paragraphs and list items into individual sentences with sequential IDs (s-0001, s-0002, …) in reading order across the whole document.
• Strip citation markers out of sentence text into each sentence's citations list.
• Leave provenance fields (authored_by, authored_at, …) absent. The document already exists; its authorship history is usually unknown. Only stamp provenance if the user volunteers it.