document-audit-extraction

Document Audit & Feature Extraction

Systematically inventory, evaluate, and extract structured data from document collections.

Audit Framework

Four-Phase Audit

Phase 1: Inventory    → What exists?
Phase 2: Classify     → What type is each document?
Phase 3: Evaluate     → What's the quality?
Phase 4: Extract      → What structured data can we pull?

Phase 1: Inventory

Automated Inventory

from pathlib import Path
from dataclasses import dataclass

@dataclass
class DocumentEntry:
    path: str
    name: str
    extension: str
    size_bytes: int
    modified: str
    word_count: int
    has_frontmatter: bool

def inventory_documents(root: str, patterns: list[str] = ["*.md", "*.txt", "*.yaml"]) -> list[DocumentEntry]:
    entries = []
    for pattern in patterns:
        for path in Path(root).rglob(pattern):
            content = path.read_text(errors="ignore")
            entries.append(DocumentEntry(
                path=str(path.relative_to(root)),
                name=path.stem,
                extension=path.suffix,
                size_bytes=path.stat().st_size,
                modified=path.stat().st_mtime,
                word_count=len(content.split()),
                has_frontmatter=content.startswith("---"),
            ))
    return entries

Inventory Report

## Document Inventory

| Path | Type | Words | Frontmatter | Modified |
|------|------|-------|-------------|----------|
| skills/dev/testing/SKILL.md | skill | 1,245 | Yes | 2026-03-20 |
| docs/CHANGELOG.md | changelog | 890 | No | 2026-03-19 |
| README.md | readme | 450 | No | 2026-03-18 |

**Total:** 142 documents | **With frontmatter:** 105 | **Total words:** 185,000

Phase 2: Classification

Document Type Taxonomy

Type	Signal	Example
Skill	YAML frontmatter with name:, in skills/	SKILL.md
Configuration	YAML/JSON schema	seed.yaml, registry.json
Guide	Tutorial structure, step-by-step	getting-started.md
Reference	API docs, schema docs	api-spec.md
Decision	ADR format, options + decision	adr-001.md
Changelog	Date-ordered entries	CHANGELOG.md
Policy	Rules, constraints	CONTRIBUTING.md