neqsim-document-intelligence-extraction

Document Intelligence Extraction

Use this skill as the intake layer for documents and images supplied to NeqSim tasks. It makes extraction automatic where tools are available without treating OCR or model output as verified engineering data.

When to Use

• A task receives PDF, DOCX, XLSX, CSV, HTML, XML, presentation, image, scan, drawing, chart, or photograph inputs.
• A PDF may contain both embedded text and visual engineering content.
• Multiple sources must be reconciled without silently choosing one value.
• Extracted values will feed simulation, design, safety, operations, or reporting workflows.

Inputs

• source_path: source file path; retain the original file unchanged.
• embedded_text_chars: optional PDF text-layer yield used to trigger OCR.
• contains_visuals: optional Office-document visual-content hint.
• Adapter-produced facts with original text, page or locator, extraction method, confidence, and optional normalized value/unit.

Outputs

• ExtractionPlan: source kind plus ordered native-text, table, OCR, rendering, and vision operations.
• ExtractionResult: schema-versioned evidence package with facts, quality score, warnings, gaps, and review status.
• Conflict records for fields whose normalized values differ across sources.

Engineering Method

The skill uses a staged, loss-minimizing method: preserve native structure, add OCR only when the text layer is absent or weak, use vision when spatial meaning matters, and reconcile all methods into evidence rather than free-form prose. Its Python package implements deterministic classification, routing, provenance validation, review gates, quality scoring, and conflict detection. Runtime adapters perform format parsing, OCR, and multimodal inference. This is an engineering-data governance workflow, not a validated physical model or a substitute for document control.

Extraction Workflow

1. Inventory files recursively and preserve original names, hashes, and relative paths.
2. Classify by actual format where possible, not filename alone. Reject encrypted, corrupt, unexpectedly executable, or unsupported content for manual triage.
3. Prefer structured/native extraction first:
   • PDF text layer and table geometry;
   • DOCX paragraphs, tables, headers, footnotes, and embedded media;
   • XLSX sheet names, cells, formulas, merged ranges, hidden rows/columns, and units;
   • CSV/JSON/XML structure without flattening it to prose.
4. For low-yield/scanned PDFs, render pages and run OCR with word coordinates. Keep OCR text separate from native text.
5. Use multimodal vision for drawings, P&IDs, charts, photographs, symbols, topology, annotations, and spatial relationships. OCR alone cannot establish these semantics.
6. Reconcile native text, OCR, tables, and vision. Do not overwrite disagreements; emit a conflict requiring review.
7. Normalize values and units while preserving the exact original value, text, and unit.
8. Gate every safety-critical, ambiguous, or confidence-below-0.85 fact as needs_review.
9. Hand the evidence package to a document-type skill for engineering interpretation, then to the relevant NeqSim model.