pdf2-md

PDF-to-Markdown: Academic Paper Conversion

Convert a PDF paper into a single Markdown file by sending pages as images to the LLM and reconstructing the full document.

Strategy Overview

1. Render each PDF page to a high-quality image (PNG, 200 DPI)
2. Send images to the LLM (yourself) page by page or in batches
3. Reconstruct the complete Markdown with correct reading order, headings, figures, tables, and LaTeX equations

Step 0 — Confirm Input

• Ask the user for the PDF path if not already provided.
• Determine output location: default to <pdf_stem>/content.md next to the PDF.

Step 1 — Render PDF Pages to Images

Use pdftoppm (Poppler) to render each page to PNG:

mkdir -p /tmp/pdf2md_pages
pdftoppm -png -r 200 "<pdf_path>" /tmp/pdf2md_pages/page

If pdftoppm is unavailable, use PyMuPDF as fallback:

import fitz
doc = fitz.open(pdf_path)
for i, page in enumerate(doc):
    pix = page.get_pixmap(dpi=200)
    pix.save(f"/tmp/pdf2md_pages/page-{i+1:03d}.png")
doc.close()

Verify image count matches expected page count.

Step 2 — Page-by-Page LLM Extraction

Read each rendered page image and extract its content. For each page, apply these rules:

2.1 Reading Order

• For two-column layouts: left column first, then right column, top to bottom.
• Full-width elements (title, abstract, full-width figures) at their natural vertical position.

2.2 Heading Hierarchy

CRITICAL: Do NOT assign heading levels by font size alone. Font size only provides initial hints. You MUST determine the final hierarchy by semantic context — the parent-child nesting of sections.

Two-pass approach:

1. First pass (per-page): Tag each heading candidate with its visual weight (font size, bold, numbering pattern) and approximate level.
2. Second pass (whole-document): After all pages are processed, review the full heading list and correct levels based on these rules:

Rules for heading level assignment:

• # — Paper title only (exactly one).
• ## — Top-level sections. Identify these by: (a) they appear in the paper's logical flow as major divisions (Abstract, Introduction, Related Work, Method, Experiments, Conclusion, Acknowledgments, References); (b) they typically share the same font size/weight; (c) numbered sections at the same depth (1, 2, 3…) are all ##.
• ### — Subsections that live inside a ## section. Identified by sub-numbering (3.1, 3.2) or smaller/different font weight under a parent.
• #### — Sub-subsections (3.1.1) or paragraph-level bold headings.