download-gated-pdfs

download-gated-pdfs is a skill for Claude Code from kennethkhoocy/legal-scholarship-skills. It costs 137 tokens per session (794 once invoked), scanned A, a copy of download-gated-pdfs, MIT.

A method for downloading the original PDF file from websites that show bot-check pages instead of the document to automated tools.

In plain words
What is it for?
Use it for PDF downloads from bot-gated think-tank, publisher, SSRN, Tax Policy Center, or Urban Institute pages.
Why use it?
It solves the problem of receiving HTML pretending to be a PDF, which causes PDF readers to report an invalid file.

Skill for Claude Code

Written for Claude Code: shipped in a Claude Code plugin. Also seen: mentions Claude Code.

Part of the legal-scholarship plugin — 7 skills shipped together

Good fit Use it for PDF downloads from bot-gated think-tank, publisher, SSRN, Tax Policy Center, or Urban Institute pages.

Compare 6 skills from other repositories ↓
Install with agentmods
npx agentmods add skills/kennethkhoocy/legal-scholarship-skills/download-gated-pdfs
Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

Any agent
npx skills add kennethkhoocy/legal-scholarship-skills --skill download-gated-pdfs
Clone the repo
git clone --depth 1 https://github.com/kennethkhoocy/legal-scholarship-skills

Made for: Claude Code.

Or install legal-scholarship, the plugin that ships this one along with the rest of its 7 skills.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for download-gated-pdfs

README.md
[![agentmods](https://agentmods.dev/badge/skills/kennethkhoocy/legal-scholarship-skills/download-gated-pdfs/github.svg)](https://agentmods.dev/skills/kennethkhoocy/legal-scholarship-skills/download-gated-pdfs)
Your own site
<a href="https://agentmods.dev/skills/kennethkhoocy/legal-scholarship-skills/download-gated-pdfs"><img src="https://agentmods.dev/badge/skills/kennethkhoocy/legal-scholarship-skills/download-gated-pdfs/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

agentmods 80×15 button for download-gated-pdfs

Your own site · 80×15
<a href="https://agentmods.dev/skills/kennethkhoocy/legal-scholarship-skills/download-gated-pdfs"><img src="https://agentmods.dev/badge/skills/kennethkhoocy/legal-scholarship-skills/download-gated-pdfs.svg" alt="Reviewed on agentmods" width="80" height="20"></a>
Per session 137 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 794 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 1 finding. A grade says what 26 rules found in the file — not that it is safe.
Origin 100% copy Near-identical to another mod in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00137 $0.00794
Opus 5 $0.00068 $0.00397
Sonnet 5 $0.00027 $0.00159
Haiku 4.5 $0.00014 $0.00079

Measured 12d ago against content hash 991e75d80f01, method: parsed. Prices are Anthropic first-party input rates as of 2026-09-12, from the pricing page.

Security

Grade A, and why

download-gated-pdfs scanned grade A with 1 finding against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 12d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Makes network callslowCapability

Not a fault in itself. Listed so you know the mod talks to something, and to what.

Use when: (1) curl/WebFetch of a .pdf URL returns HTML instead of a PDF even with a
Origin

This is a copy

100% identical to download-gated-pdfs — 0 lines differ, which has more behind it and is treated as the original. This page carries a canonical link to it rather than competing with it.

plugins/legal-scholarship/skills/download-gated-pdfs/SKILL.md · 64 lines

How it starts

The opening of the file, as written. The whole thing — 64 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Download bot-gated PDFs via Wayback id_

Problem

Many think-tank and publisher sites (taxpolicycenter.org, urban.org, SSRN delivery) serve an HTML bot-challenge page instead of the PDF to non-browser clients. A browser User-Agent header does not help. The downloaded "PDF" is actually HTML.

Context / Trigger Conditions

  • curl -o file.pdf <url> succeeds but the file starts with <!DOC
  • pypdf raises invalid pdf header: b'<!DOC' or PdfStreamError: Stream has ended unexpectedly
  • Firecrawl scrape returns clean markdown for the same URL (its proxies get through), but Firecrawl does not return the binary — only parsed content

Solution

  1. Request the file through the Wayback Machine's raw-content (id_) endpoint, which serves the original archived binary without rewriting:
    curl -sL -A "Mozilla/5.0 ... Chrome/126.0 Safari/537.36" \
      "https://web.archive.org/web/<YYYY>id_/<original-pdf-url>" -o out.pdf
    
    <YYYY> is any year likely to have a snapshot (e.g. publication year); Wayback redirects to the nearest capture. The id_ suffix after the timestamp is what requests the untouched original.
  2. Verify the download with pypdf — a bot page fails immediately:
    from pypdf import PdfReader
    r = PdfReader("out.pdf"); print(len(r.pages), "pages")
    
  3. If Wayback has no capture, fall back to: another mirror found via search (Exa/Firecrawl), or Firecrawl scrape for the parsed text when the binary is not strictly needed.

Verification

PdfReader opens the file and reports a plausible page count; first-page text matches the expected title.

Example

Verified 2026-07-15: taxpolicycenter.org/sites/default/files/publication/165884/ssrn-id4797771.pdf and urban.org/sites/default/files/publication/80621/2000790-...pdf both bot-gated to direct curl (with UA), both downloaded intact via https://web.archive.org/web/2024id_/<url> and .../web/2023id_/<url> (18 and 12 pages).

Notes

  • Government data hosts (e.g. ticdata.treasury.gov) are usually NOT gated — try direct curl first; Wayback is the fallback, not the default.
  • Wayback captures can be stale for frequently-revised documents; check the snapshot date if currency matters.
  • See also: the pdf skill (parsing/extraction after download) and firecrawl:firecrawl-scrape (parsed markdown when the binary is unnecessary).

Read the full file on GitHub · 64 lines

Files

What ships with it

1 file beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 12d ago First seen · 64 lines · 137 tokens per session scan A 991e75d80f01

Subscribe to this mod's changes

download-gated-pdfs is a skill published in the GitHub repository kennethkhoocy/legal-scholarship-skills (9 stars, last pushed 7d ago), licensed MIT. It adds 137 tokens to every session and 794 once invoked, about $0.0007 per session on Opus 5. A static security scan graded it A with 1 finding (makes network calls). It is 100% identical to download-gated-pdfs, differing in 0 lines, and is treated as a copy.

Related

Other skills, from other repositories

download-fulltext-pdf

A skill for downloading a research paper's complete PDF using an identifier such as a DOI, title, or BibTeX entry.

huangwb8/skills · 105 tokens

markdown-to-pdf

Convert Markdown (.md) files to a polished PDF with ALL images preserved and scaled to the page. Use whenever the user asks to "save this markdown as a PDF", "convert README.md to pdf", "export the .md as a pdf", "turn these notes/docs into a PDF", or wants a PDF deliverable of any GitHub-flavored Markdown document…

kennethkhoocy/applied-micro-skills · 188 tokens

report-helper

A Chinese-language research workflow that searches the internet and produces a formatted PDF report about a specified topic.

Jiaranbb/report-helper · 92 tokens

download-gated-pdfs

Download the actual PDF binary from bot-gated sites (taxpolicycenter.org, urban.org, SSRN-hosted mirrors, think-tank/publisher sites) via the Wayback Machine id URL form. Use when: (1) curl/WebFetch of a .pdf URL returns HTML instead of a PDF even with a browser User-Agent, (2) pypdf fails with "invalid pdf header…

kennethkhoocy/applied-micro-skills · 137 tokens

pdf-analyze

Comprehensive PDF manipulation toolkit for extracting text and tables, creating new PDFs, merging/splitting documents, and handling forms. When Claude needs to fill in a PDF form or programmatically process, generate, or analyze PDF documents at scale.

aiskillstore/marketplace · 52 tokens

pdf-processing

Extract text and tables from PDF files, fill forms, merge documents. Use when working with PDF files or when the user mentions PDFs, forms, or document extraction.

aiskillstore/marketplace · 36 tokens