document-processing-expert

Document Processing Expert

Working with the formats people actually exchange. The recurring mistake is treating a document as text: these are structured containers, and the structure is where the meaning lives.

Core Concepts

Read the Format, Not the Bytes

.docx, .xlsx and .pptx are ZIP archives of XML parts. PDF is a page description language, not a document model. Using a library that understands the format is not a convenience — string manipulation on these files produces output that opens corrupted.

Extraction Is Lossy; Decide What Matters

Every extraction discards something: layout, styling, reading order, or the relationship between a heading and its table. Decide up front whether you need text, structure, or fidelity, and choose the tool accordingly.

Generation Is About Templates

Building a document element by element in code produces unmaintainable output that a designer cannot touch. Fill a template that a human authored, in the tool they authored it in.

Untrusted Documents Are Untrusted Input

Documents carry macros, external entities, embedded files and remote references. A parsing library is an attack surface. Treat every uploaded document as hostile.

PDF

Extracting text

import pdfplumber

def extract_pages(path: str) -> list[str]:
    with pdfplumber.open(path) as pdf:
        return [page.extract_text() or "" for page in pdf.pages]

pdfplumber preserves layout well and extracts tables; pypdf is faster for plain text and page manipulation. If extract_text() returns empty on a page that clearly has content, the page is an image — you need OCR, not a different parser.

Extracting tables

with pdfplumber.open(path) as pdf:
    for page in pdf.pages:
        for table in page.extract_tables():
            header, *rows = table
            records = [dict(zip(header, row)) for row in rows]

Table extraction is unreliable on borderless or multi-page tables. Validate the result — column count, expected types, totals that reconcile — rather than trusting it.