safe-markdown-auto-fix

Safe Markdown auto-fix

When to use

You're writing (or reviewing) a regex-based fixer for Markdown — spacing rules, typography, terminology normalization. Applies to any lint-and-fix tool, including the cndocstyle package shipped with cn-doc-style-guide.

Problem

A naive re.sub over the whole file will cheerfully mangle:

• Fenced code blocks ( ): inserting "helpful" spaces in the middle of a Python string breaks the program.
• Inline code spans (`foo_bar`): adding space around _ ruins identifiers.
• Link URLs ([text](https://…)): even reading URLs as prose produces false positives like "space between CJK and ASCII" inside a percent-encoded path.
• HTML tags / comments: <img alt="你好World"> is prose inside HTML, but the tag itself must stay intact.

Solution

Always tokenize the file into "protected" and "prose" regions before applying any substitution. A minimal protection list:

PROTECT = [
    # fenced code blocks, greedy across lines
    re.compile(r"```[\s\S]*?```", re.MULTILINE),
    # indented code blocks (4+ leading spaces at start of line)
    re.compile(r"(?:^|\n)(?: {4,}|\t).+", re.MULTILINE),
    # inline code
    re.compile(r"`[^`\n]+`"),
    # link / image URLs — keep the URL, allow fixing the text
    re.compile(r"\]\([^)\n]+\)"),
    # raw HTML tags and comments
    re.compile(r"<!--[\s\S]*?-->"),
    re.compile(r"<[a-zA-Z/][^>]*>"),
]

Then:

1. Walk the text, carving out every match into a placeholder (e.g. \x00PROTECT_N\x00) and storing the original.
2. Run your prose-level substitutions on the placeholder-laced text.
3. Restore placeholders.

For a fixer it's also important to:

• Default to dry-run. Print a unified diff; require --apply to write.
• Be idempotent. Running the tool twice must not change output.
• Fail closed. If in doubt (ambiguous context, mixed-script identifiers), leave the text untouched and log a warning rather than silently "correcting" it.