parser-development

Parser Development

Use crates/biome_parser/CONTRIBUTING.md as the canonical parser guide. Read the section matching the current grammar, lexer, parse-rule, or recovery task.

Workflow

1. Inspect the language grammar and neighboring parse rules.
2. Add or adjust focused ok/ and error/ fixtures before changing recovery behavior.
3. Update the grammar when the typed tree shape changes.
4. Regenerate syntax and factory artifacts after .ungram changes.
5. Implement parser logic with explicit presence tests and bounded recovery.
6. Run the language parser's focused tests and inspect snapshots.

Grammar

Grammar files under xtask/codegen/ define typed syntax nodes and fields. Follow existing language naming:

• prefix nodes with the language name;
• name unions with Any;
• use Bogus nodes for recoverable invalid syntax;
• end list node names with List;
• represent lists as present, possibly empty nodes rather than optional fields.

After a grammar change, run:

just gen-grammar <lang>

This updates generated syntax nodes, syntax kinds, factories, macros, and language-specific mappings. Parser rules remain hand-written.

Presence Contract

A parse function returns Absent only when it has consumed no tokens. Test the first distinguishing token before calling start, bump, eat, expect, or another parser that can advance.

Use:

• expect for required tokens that should create a diagnostic when absent;
• eat for optional tokens;
• .ok() for optional nodes;
• .or_add_diagnostic(...) for required nodes;
• typed recovery for malformed nodes that should remain in the CST.

Do not use backtracking where a bounded lookahead or a more precise presence test can distinguish the syntax.

Error Recovery

Recovery must preserve following valid syntax and produce a bogus node permitted by the grammar at that position.

A recovery set normally includes the nearest relevant:

• list separator;
• list or block terminator;
• statement boundary;
• token that starts the next valid construct.