insight-knowledge-harvest

Insight Knowledge Harvest

Use this skill to build and incrementally expand a curated raw-material layer for future viewpoint and insight extraction.

This is a source-ingestion and verification workflow, not a final insight-writing workflow. Its job is to inspect the current KB as context, promote or discover candidate page-level sources, deduplicate them, capture accepted pages into raw Markdown, and keep ingest plus verification state aligned across the project.

The operating model is intentionally simple:

• source/raw/ stores one raw page capture per canonical source page or document, plus minimal identity, processing-state, and high-level classification metadata.
• source/ingest.md tracks minimal pipeline state for every candidate or accepted material.
• source/registers/ holds curator-facing notes such as rejection reasons, gap lists, deep-read queues, and the classification schema vocabulary.
• source/.harvest/ optionally stores hidden operational deduplication state and detailed classification metadata in SQLite for broader discovery runs.

In default additive mode, existing materials are treated as read-only context for topic promotion, deduplication, and gap detection. The skill normally adds new page-level captures, verifies newly downloaded files in place, and avoids rewriting prior raw materials unless the user explicitly asks for repair or re-verification.

Raw files must remain raw captures, not summaries. Curator judgment belongs in ingest, register notes, or the internal SQLite index, while the raw file keeps only source identity, processing-state metadata, and the high-level classification trio: material_kind, topic_domain, and credibility_tier.

At a glance, this skill is best when you need to:

• grow an evidence base before insight extraction,
• keep one-file-per-page provenance clean,
• run incremental source discovery without reprocessing the whole archive,
• and preserve a separation between raw source content and curator-written interpretation.