generate-scraper

generate-scraper is a skill for Claude Code from ScrapeOps/scrapeops-scraping-assistant-claude-plugin. It costs 225 tokens per session (17,046 once invoked), scanned B, original, MIT.

A generator for single-URL web scrapers: programs that fetch a webpage and extract selected information into JSON Lines, a text format with one JSON record per line.

In plain words
What is it for?
Use it when you have specific URLs and want a generated scraper that runs through ScrapeOps, reads its API key from the environment, and saves extracted results.
Why use it?
It removes the need to write the scraper structure, proxy setup, command-line input, and usage documentation from scratch.

Skill for Claude Code

Written for Claude Code: shipped in a Claude Code plugin. Also seen: reads .claude/ paths; names the AskUserQuestion tool.

Needs its repository: it runs a file that does not travel with it, so clone the repository first. The line is node src/main.js --url "<first user URL>".

Part of the scrapeops-scraping-assistant-claude-plugin plugin — 5 skills, 1 agent shipped together

Good fit Use it when you have specific URLs and want a generated scraper that runs through ScrapeOps, reads its API key from the environment, and saves extracted results.

Compare 6 skills from other repositories ↓
Install

Getting it into your agent

It runs from inside its repository, so the clone comes first — what it calls does not travel with the file alone.

Clone the repo
git clone --depth 1 https://github.com/ScrapeOps/scrapeops-scraping-assistant-claude-plugin
agentmods
npx agentmods add skills/scrapeops/scrapeops-scraping-assistant-claude-plugin/generate-scraper

Made for: Claude Code.

Or install scrapeops-scraping-assistant-claude-plugin, the plugin that ships this one along with the rest of its 5 skills, 1 agent.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for generate-scraper

README.md
[![agentmods](https://agentmods.dev/badge/skills/scrapeops/scrapeops-scraping-assistant-claude-plugin/generate-scraper.svg)](https://agentmods.dev/skills/scrapeops/scrapeops-scraping-assistant-claude-plugin/generate-scraper)
Your own site
<a href="https://agentmods.dev/skills/scrapeops/scrapeops-scraping-assistant-claude-plugin/generate-scraper"><img src="https://agentmods.dev/badge/skills/scrapeops/scrapeops-scraping-assistant-claude-plugin/generate-scraper.svg" alt="Measured on agentmods" height="20"></a>
Per session 225 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 17,046 The whole file, excluding the scripts and references it only reads on demand.
Security scan B 2 findings. A grade says what 26 rules found in the file — not that it is safe.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00225 $0.17046
Opus 5 $0.00112 $0.08523
Sonnet 5 $0.00045 $0.03409
Haiku 4.5 $0.00022 $0.01705

Measured 8d ago against content hash 2d42b558a5ca, method: parsed. Prices are Anthropic first-party input rates as of 2026-09-08, from the pricing page.

Security

Grade B, and why

generate-scraper scanned grade B with 2 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 8d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Reads agent configuration directoriesmediumAgent snooping

.claude/, .codex/, .gemini/ hold keys, settings and other credentials a mod has no legitimate need for.

> **API key handling:** The MCP server reads `SCRAPEOPS_API_KEY` from `~/.claude/settings.json` automatically. Do NOT check or validate it before making tool calls. If a MCP tool call fails with "SCRAPEOPS_API_KEY is not

Makes network callslowCapability

Not a fault in itself. Listed so you know the mod talks to something, and to what.

Download the canonical framework docs to `/tmp/`. **Use `Bash curl` with a browser
skills/generate-scraper/SKILL.md · 1,448 lines

How it starts

The opening of the file, as written. The whole thing — 1,448 lines — stays where its author put it; the contents beside it link to each section on GitHub.

ScrapeOps — Generate Scraper Skill

Generates a production-ready single-URL scraper using the ScrapeOps AI backend via the scrapeops MCP server. The generated script:

  • Accepts a URL as a CLI argument at runtime.
  • Fetches HTML in-memory through the ScrapeOps proxy (https://proxy.scrapeops.io/v1/ for HTTP-client libs, or residential-proxy.scrapeops.io:8181 for browser-based libs).
  • Runs the extractor and writes the result to a JSONL output file.
  • Reads the API key from the SCRAPEOPS_API_KEY environment variable.
  • Ships with a README.md that explains install, env setup, and usage.

No HTML is downloaded to disk at generation time. No execution happens at the end — the user runs the scraper manually against any URL they want.

Overview

  1. Collect inputs (URLs, language, library)
  2. Submit job via scrapeops_submit_job with parser_only: false + include_api_key: false
  3. Poll until complete via scrapeops_poll_status
  4. Retrieve the full scraper via scrapeops_get_code
  5. Install dependencies (optional)
  6. Save the file verbatim, then apply two targeted Edits: (a) API-key env var, (b) hardcoded URL list → CLI arg
  7. Generate <domain>_README.md
  8. Show the final summary — do NOT execute the scraper

API key handling: The MCP server reads SCRAPEOPS_API_KEY from ~/.claude/settings.json automatically. Do NOT check or validate it before making tool calls. If a MCP tool call fails with "SCRAPEOPS_API_KEY is not configured", handle it inline:

  1. Ask the user: "Enter your ScrapeOps API key (find it at https://scrapeops.io/app/dashboard):"
  2. Read ~/.claude/settings.json, add the key to env.SCRAPEOPS_API_KEY using Edit, and retry the failed tool call
  3. Do NOT tell the user to run /scrapeops-setup or restart — just ask, save, and continue

Step 1 — Collect Inputs

Slot Required
urls Yes — 1 to 5 URLs from the same domain. They are used ONLY as generation-time samples to teach the backend the page structure. At runtime the generated script accepts a single URL via CLI.
language Yes — always ask if not explicitly provided
library Yes — always ask if not explicitly provided
country No — skip if not mentioned

Read the full file on GitHub · 1,448 lines

Files

What ships with it

3 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 8d ago First seen · 1,448 lines · 0 tokens per session scan B 2d42b558a5ca

Subscribe to this mod's changes

generate-scraper is a skill published in the GitHub repository ScrapeOps/scrapeops-scraping-assistant-claude-plugin (5 stars, last pushed 4mo ago), licensed MIT. It adds 225 tokens to every session and 17,046 once invoked, about $0.0011 per session on Opus 5. A static security scan graded it B with 2 findings (reads agent configuration directories, makes network calls). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.

Related

Other skills, from other repositories

opencli-sitemap-author

Use when creating or maintaining OpenCLI site sitemaps: agent-facing navigation, page-state, action, workflow, API-reference, pitfall, and fallback knowledge for a website. Use after browser exploration discovers durable site context, when a sitemap is stale, or when promoting local site knowledge into the repo.

jackwener/OpenCLI · 67 tokens

debug-optimize-lcp

Guides debugging and optimizing Largest Contentful Paint (LCP) using Chrome DevTools MCP tools. Use this skill whenever the user asks about LCP performance, slow page loads, Core Web Vitals optimization, or wants to understand why their page's main content takes too long to appear. Also use when the user mentions…

ChromeDevTools/chrome-devtools-mcp · 99 tokens

interactive-login

How to complete browser/interactive logins (aws / gh / glab / gcloud). The platform backgrounds the login poller so it survives the human's browser round-trip — and when that does NOT work.

yc-software/qm · 46 tokens

pinchtab-mcp

Use this skill when a task requires browser automation through PinchTab's MCP server connected to a remote browser instance. Covers navigation, element interaction, data extraction, form filling, multi-step flows, and session management via MCP tools.

pinchtab/pinchtab · 52 tokens

azure-messaging-webpubsub-java

Build real-time web applications with Azure Web PubSub SDK for Java. Use when implementing WebSocket-based messaging, live updates, chat applications, or server-to-client push notifications.

microsoft/skills · 43 tokens

google-safe-browsing

Prevent and fix Google Safe Browsing "Dangerous site" flags. Use when launching a public web app, buying/picking a domain, building a login or signup page, or when any site shows a red "Dangerous site" / "Deceptive site" warning in Chrome, Brave, Safari, Firefox, or Edge. Triggers on "dangerous site", "deceptive…

davidondrej/skills · 105 tokens