obsidian-ingest

Use the obsidian-second-brain skill. Execute /obsidian-ingest $ARGUMENTS:

The argument is a URL, file path, or pasted text. If no argument, ask what to ingest.

1. Read _CLAUDE.md first if it exists in the vault root

2. Classify the source type before reading the full content:

   • Article/blog post - extract key claims, people, tools, concepts
   • PDF/document - extract structure, findings, recommendations
   • Transcript (meeting/podcast) - extract speakers, decisions, action items, quotes
   • YouTube video - pull metadata, description, and transcript (see step 3 for method)
   • Audio file (.m4a, .mp3, .wav, .ogg, .webm) - transcribe, identify speakers, extract decisions/tasks/promises
   • Image/screenshot (.png, .jpg, .jpeg, .webp) - read/OCR the image, extract text and context
   • Raw text - classify by content (opinion, technical, narrative) and extract accordingly

3. Read or fetch the full source content:

   For YouTube URLs - try methods in this order (use the first one that works):

   Method A - yt-dlp (best, works in Claude Code / terminal):

   which yt-dlp || brew install yt-dlp
   yt-dlp --skip-download --print title --print description --print duration_string --print view_count --print like_count --print upload_date --print channel "URL"
   yt-dlp --write-auto-sub --sub-lang en --skip-download -o "/tmp/%(id)s" "URL"
   

   Method B - YouTube MCP tools (works in Claude Desktop if configured): Check if YouTube MCP tools are available. If so, use them.

   Method C - oEmbed fallback (works everywhere, limited data): Fetch https://www.youtube.com/oembed?url=URL&format=json - gives title and channel only. Ask user to paste description for full ingest.

   For audio files (.m4a, .mp3, .wav, .ogg, .webm):

   # Transcribe with Whisper (install if missing)
   which whisper || pip install openai-whisper
   whisper "path/to/audio.m4a" --model base --output_format txt --output_dir /tmp
   

   If whisper can't be installed, ask the user to paste the transcript. After transcription: identify speakers if possible, extract decisions, action items, promises, and who said what. Save the transcript to raw/transcripts/.

   For images/screenshots (.png, .jpg, .jpeg, .webp): Claude can read images directly. Analyze the image for:

   • Text content (OCR) - extract all readable text
   • UI screenshots - describe what's shown, extract data from tables/forms/dashboards
   • Whiteboard/diagram photos - describe the structure and extract concepts
   • Chat screenshots - extract messages, people, decisions Save the image description to raw/articles/ as a markdown summary with context.

   For articles - use the WebFetch tool to pull the page content For PDFs - read the file directly For pasted text - use as-is

4. Extract and organize:

   • Entities: people mentioned, companies, tools, projects
   • Concepts: key ideas, frameworks, methodologies
   • Claims: specific assertions with supporting evidence
   • Action items: anything actionable for the user
   • Quotes: notable quotes worth preserving

5. Save the raw source to raw/ (immutable - never modify after saving):

   • Check for a previous ingest of this source first (#218, #239). Compute content_hash over the canonical text of the source, never over the raw capture: the same page fetched twice rarely yields the same bytes (a JS shell one time and rendered DOM the next, navigation chrome, a cookie banner, a + where the page had -), and a hash over those bytes calls an unchanged source "changed" on every run. Canonicalize in this order, then hash: (1) keep the article body only - drop navigation, header, footer, sidebar, cookie and consent banners, share widgets, comment sections, and any frontmatter or metadata block the page itself embeds; (2) convert CRLF to LF and drop a leading BOM; (3) normalize list markers * and + to -; (4) collapse every run of whitespace, newlines included, to one space and trim. content_hash is the first 16 hex characters of the SHA-256 of that string (printf '%s' "$CANONICAL" | shasum -a 256 | cut -c1-16). The raw note body stays the verbatim capture; only the hash is computed over the canonical form. Then search raw/ frontmatter for that content_hash, and for the same source_url (normalized: strip the scheme, www., trailing slash and tracking parameters such as utm_*). Use Grep, not memory.
     • Same hash found: the source is already in the vault. Do not write a second raw note. Skip to step 6 and treat this run as a re-read: build the proposals from the existing raw note, and say in the report that the source was already ingested on the date in its frontmatter. A re-read is bound by the confirmation rule in step 6 like any other run: it may not rewrite an existing note without the user's yes.
     • Same URL, different hash: diff the new canonical text against the canonical form of the stored raw note before deciding. If the delta is capture noise the canonicalization missed (chrome, whitespace, list markers), treat it as the same-hash case: no second raw note, nothing superseded, and the stored note keeps its hash (raw notes are immutable; the URL match is what finds it next time as well). If the article text itself changed, the source changed since it was last ingested: write the new raw note, add supersedes: "[[<old raw note>]]" to its frontmatter, and in step 6 give the Contradictions agent the old raw note as well, because claims that came from the old version may now be stale.
     • Neither found: this is the first ingest. Proceed.
   • Create raw/articles/YYYY-MM-DD - Source Title.md (or transcripts/, pdfs/, videos/)
   • Frontmatter: type: source, date, tags: [source, <type>], source_url, source_type, content_hash, ai-first: true (the raw-source schema in references/ai-first-rules.md; the body stays verbatim - preamble not required)