pipeline

/guide:pipeline — Daily Content Pipeline

"Time is an illusion. Lunchtime doubly so." — But your daily brief runs on schedule.

Run the full content pipeline: fetch signals, analyze relevance, create daily brief + drafts, edit for voice fidelity, and deliver output with notifications.

Prerequisites

• content/config.yaml in the user's project (falls back to plugin defaults if missing)
• Python 3.10+ with feedparser and pyyaml
• jq and curl on PATH
• Optional: gws CLI for Gmail, osascript for iMessage

---

Phase 0: Config Check

Before anything else, check if content/config.yaml exists.

• If it exists: Read it and proceed to Phase 1.
• If it doesn't exist: Tell the user: "No config found. Run /guide:setup to configure your content pipeline — it takes about 2 minutes and I'll walk you through it." Then stop. Do NOT use the defaults config as a silent fallback for a first run — the user needs to configure their own sources and themes.

---

Phase 1: Scout (Fetch Sources)

Run the pipeline fetch script to gather all external signals deterministically.

Finding Scripts

The fetch scripts live in this plugin's scripts/ directory. Determine the scripts path:

• If this plugin is at heart-of-gold-toolkit/plugins/guide/, use heart-of-gold-toolkit/plugins/guide/scripts/
• If installed via marketplace, use the plugin installation path's scripts/ directory
• Locate by searching for fetch-rss.py in the project tree if unsure

Steps

1. Read config from content/config.yaml
2. Read voice reference from the path in voice.reference config field
3. Run the pipeline fetcher — a single deterministic script that calls ALL configured sources (RSS, Gmail, HN, Events/iCal), combines them, normalizes scores, and writes output:

   bash <scripts>/run-pipeline-fetch.sh --config content/config.yaml
   

   This script:
   • Calls each enabled source (fetch-rss.py, fetch-gmail.sh, fetch-hn.sh, fetch-events.py)
   • Combines all signals into a single array
   • Normalizes scores to 0.0-1.0 with source weight multipliers applied
   • Writes content/pipeline/YYYY-MM-DD/signals.json (with collision avoidance: signals-2.json, etc.)
   • Writes content/pipeline/YYYY-MM-DD/fetch-log.json with per-source status
   • Do NOT run individual fetch scripts yourself. The runner handles all of them.
4. Read signals.json from the pipeline directory — this is your input for Phase 2
5. Read fetch-log.json — check which sources succeeded/failed. If a source failed, mention it in the brief footer.
6. Read captures from content/captures/ (or configured captures_dir) — last 7 days of AM/PM captures
7. Read recent daily briefs — last 3 briefs from content/daily/ for deduplication context