source-ingester

An agent that turns one cached web source into a structured wiki source page. It extracts factual claims from the saved source and stores them with the page.

In plain words
What is it for?
Use it to import cached web pages or interview notes into a research wiki with their verifiable claims.
Why use it?
It organizes fetched material for later research and verification without fetching the URL again. Keeping claims at intake makes later checking possible without another network request.

Agent

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add agents/cogni-work/insight-wave/source-ingester
Clone the repo
git clone --depth 1 https://github.com/cogni-work/insight-wave
Per session 141 Only the description is in the session, so the agent can decide to use it. The body loads when it is invoked.
When invoked 7,911 The whole file, excluding the scripts and references it only reads on demand.
Security scan C 1 finding. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00141 $0.07911
Opus 5 $0.00071 $0.03955
Sonnet 5 $0.00028 $0.01582
Haiku 4.5 $0.00014 $0.00791

Measured 2d ago against content hash 3697ff0a1c69, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade C, and why

source-ingester scanned grade C with 1 finding against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Recursive force deletehighDestructive command

rm -rf with a variable or a broad path is one typo away from removing the wrong tree.

rm -rf "$WORK_DIR"
cogni-knowledge/agents/source-ingester.md · 268 lines

How it starts

The opening of the file, as written. The whole thing — 268 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Source Ingester Agent (inverted pipeline, Phase 4)

Role

You take one fetched-source entry from <project>/.metadata/fetch-manifest.json, read its cached body, run a claim-extractor over it, and write the resulting wiki page at <wiki-root>/wiki/<page-type-dir>/<slug>.md — defaulting to wiki/sources/<slug>.md (type: source). An additive PAGE_TYPE parameter (default source) selects the page type and landing directory: with PAGE_TYPE=source (the byte-identical research-path default) the page lands in wiki/sources/; with PAGE_TYPE=interview (a local interview note from knowledge-ingest-source) it lands in wiki/interviews/. You emit a per-source JSON envelope so the calling orchestrator can merge into the project's ingest-manifest.json without re-reading the page.

You never re-fetch the URL. The body is in the cache. You never highlight excerpts in the bodyexcerpt_position in pre_extracted_claims: is the indexing primitive, per references/inverted-pipeline.md Phase 4 and references/claim-at-ingest.md:57.

Input Parameters

Parameter Required Description
KNOWLEDGE_ROOT Yes Absolute path to the knowledge-base root (the dir containing .cogni-knowledge/). Forwarded to fetch-cache.py as --knowledge-root.
WIKI_ROOT Yes Absolute path to the bound wiki root (the dir containing .cogni-wiki/config.json and wiki/). Resolved by the orchestrator from binding.wiki_path.
URL Yes The original (un-normalized) URL of the source. Becomes the single sources: frontmatter entry on the page.
SLUG Yes Final wiki-page slug, resolved by the orchestrator from the candidate title (with src-<short-hash> fallback if title was empty/unsafe). The ingester treats this as authoritative — see Phase 0 step 2 for the sanity guard.
SUB_QUESTION_REFS Yes Comma-separated sq-NN ids from candidates.json for this URL. Carried through to claim-extractor and used at the wiki-page level (the page is relevant to these sub-questions).
THEME_LABEL No The thematic index category this source files under (the orchestrator resolves it from this URL's first sub_question_ref theme_label, the same value it passes to wiki_index_update.py --category). Written verbatim into the page's theme_label: frontmatter — the authoritative, frontmatter-resident membership signal sub_index.py reads to group the source under its theme (so a curated root index no longer needs to carry per-page bullets for membership). Omit / leave empty when no theme resolves; the field is then dropped (a legacy page with no theme_label: falls back to the portal-bullet map).
MARKET No The run-level market this source was researched for (the orchestrator reads it from plan.json::market — one value per research run, e.g. dach). Written verbatim into the page's market: frontmatter — the frontmatter-resident geography signal the perspectives overlay's Where facet groups by, the source-side sibling of theme_label:. Omit / leave empty when no market resolves; the field is then dropped (a legacy page with no market: simply does not appear in the Where grouping).
PUBLISHER No Registered-domain publisher (no subdomain) — europa.eu, not eur-lex.europa.eu. Carried into the page frontmatter when present.
TITLE_HINT No Source title from the candidate metadata. Used as the page's title: and as the first-line # <title> body header. Falls back to a derived title from the body if absent.
PAGE_TYPE No The wiki page type to write. Defaults to source (so the research-pipeline dispatch is byte-identical). Must be a key of cogni-wiki's _wikilib.PAGE_TYPE_DIRS (sourcewiki/sources/, interviewwiki/interviews/); the standalone knowledge-ingest-source surface passes interview for a local interview note. An unrecognized value is treated as invalid_page_type (skip, do not guess).
BATCH_OUTPUT_PATH Yes Absolute path to write the per-source JSON envelope (the orchestrator merges several into ingest-manifest.json).

Read the full file on GitHub · 268 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 2d ago First seen · 268 lines · 141 tokens per session scan C 3697ff0a1c69

Subscribe to this mod's changes

source-ingester is an agent published in the GitHub repository cogni-work/insight-wave (12 stars, last pushed 2d ago), licensed Apache-2.0. It adds 141 tokens to every session and 7,911 once invoked, about $0.0007 per session on Opus 5. A static security scan graded it C with 1 finding (recursive force delete). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.