fetch-paper

Fetch Paper

Resolves one DOI or arXiv ID to a legal open-access copy (PDF URL, arXiv HTML rendering, landing page) and optionally downloads it for transient reading. This is the fetch layer that find-papers, literature-review, study-exemplars, and draft-related-work build on.

When to use

• The user has a DOI, doi.org URL, arXiv ID, or arxiv.org URL and wants the paper.
• Another skill needs full text for a paper it found (lit review, exemplar study).
• The user asks "is there an open-access version of this paper?"
• NOT for topic/title/venue search — run find-papers first to get the identifier.

Inputs

• Exactly one identifier per fetch: a DOI (10.1145/3589132.3625571, https://doi.org/...) or an arXiv ID (2403.12345, arXiv:2403.12345v2, cs/0309136, or any arxiv.org URL). arXiv-issued DOIs (10.48550/arXiv.*) are auto-converted.
• UNPAYWALL_EMAIL (or CONTACT_EMAIL) env var — a real address; Unpaywall rejects placeholders with HTTP 422. The script prompts if unset.

Process

1. Resolve the identifier. Run:

   python3 scripts/resolve_oa.py <DOI-or-arXiv-ID> --json
   

   The script handles everything deterministic: identifier normalization, Unpaywall best_oa_location lookup, arXiv PDF/HTML URLs, the post-2026 ACM DL open-access fallback for 10.1145/* DOIs, rate limiting, 429 backoff, and response caching under .cache/fetch-paper/. Never hand-roll these API calls.

2. Interpret the result (exit codes: 0 = OA found, 3 = no legal OA copy, 2 = bad input/config, 1 = network failure):

   • Prefer html_url (arXiv HTML) when present — easiest to read directly.
   • Otherwise use pdf_url. Read the meaning of oa_status, version, and license in references/oa-sources.md.
   • version: submittedVersion means a preprint — warn the user that the published version may differ before they quote or cite page numbers.

3. Fetch the full text transiently. Either read the URL directly, or: