update-references

Update references (research paper search)

This skill turns "find me what's been written about this" into a small, repeatable pipeline: read what the user is working on, search arXiv (and, for off-arXiv work, Semantic Scholar or Google Scholar) for related papers, download the relevant ones, read them, write a summary per paper, and keep a Citation Style Language (CSL) YAML database so work is never repeated and versions stay current.

The standing interests are LLMs, AI agents, finite state machines, and declarative agent patterns — the territory of spindle (github.com/petar-djukic/spindle), a Go state-machine engine for agentic loops. Lean toward those topics, but always sharpen the search using whatever the current working directory is actually about.

Where things live

This skill is context-aware. It runs from the directory the user is working in, and it stores its outputs there — not in a fixed global location. Resolve paths like this:

• Database: references.yaml under the current working directory, unless an existing one is already nearby (look for a references.yaml at or above the working directory and reuse it). Pass the chosen path as --db to every script call so search, fetch, and record all agree. If only a legacy arxiv/papers.yaml exists, the scripts read it transparently and convert on next write.
• PDFs: <db-dir>/pdfs/ (transient — they exist to be read; they don't need to be committed). The script resolves this relative to the database, not the current directory, so PDFs land next to the db even when run from elsewhere.
• Papers (markdown): <db-dir>/papers/ — fetch converts each PDF to markdown automatically (best-effort via pymupdf4llm), preserving headings, tables, and math. Papers are readable in any editor or tool.
• Summaries: <db-dir>/summaries/, one file per paper.

The PDF, markdown, and summary of a paper share one human-friendly stem: <Family>-<Year>-<title-slug>-<source>-<id> (for example Lee-2026-meta-harness-end-to-end-optimization-arxiv-2603.28052). The <source>-<id> tail is arxiv-<id>vN, doi-<doi>, or scholar-<key>, so any file traces back to its source at a glance. This differs from the pandoc citation key, which stays short — just <family>-<year> (see the database format). repair renames files created under the older naming scheme.

• Manual-download list: <db-dir>/downloads-needed.md — regenerated by scholar.py pending, a checklist of papers that could not be downloaded (paywalled or behind a login) with a landing URL for each. Derived from the database, so it is disposable, not hand-maintained. An entry carrying download: deferred is kept off the checklist (it lists compactly at the bottom instead) — set it on metadata-only entries the current work does not cite, and remove it when a citation pass starts citing the entry.