firecrawl-scraper

Firecrawl Scraper Skill

Trigger Conditions & Endpoint Selection

Choose Firecrawl endpoint based on user intent:

• scrape: Need to extract content from a single web page (markdown, html, json, screenshot, pdf)
• crawl: Need to crawl entire website with depth control and path filtering
• map: Need to quickly get a list of all URLs on a website
• batch-scrape: Need to scrape multiple URLs in parallel
• crawl-status: Given crawl job ID, check crawl progress/results (optional --wait)
• batch-status: Given batch job ID, check batch scrape progress/results (optional --wait)
• batch-errors: Given batch job ID, retrieve batch scrape errors

Recommended Architecture (Main Skill + Sub-skill)

This skill uses a two-phase architecture:

1. Main skill (current context): Understand user question → Choose endpoint → Assemble JSON payload
2. Sub-skill (fork context): Only responsible for HTTP call execution, avoiding conversation history token waste

Execution Method

Use Task tool to invoke firecrawl-fetcher sub-skill, passing command and JSON (stdin):

Task parameters:
- subagent_type: Bash
- description: "Call Firecrawl API"
- prompt: cat <<'JSON' | node scripts/firecrawl-api.cjs <scrape|crawl|map|batch-scrape|crawl-status|batch-status|batch-errors> [--wait]
  { ...payload... }
  JSON

Payload Examples

1) Scrape Single Page

cat <<'JSON' | node scripts/firecrawl-api.cjs scrape
{
  "url": "https://example.com",
  "formats": ["markdown", "links"],
  "onlyMainContent": true,
  "includeTags": [],
  "excludeTags": ["nav", "footer"],
  "waitFor": 0,
  "timeout": 30000
}
JSON

Available formats:

• "markdown", "html", "rawHtml", "links", "images", "summary"
• {"type": "json", "prompt": "Extract product info", "schema": {...}}
• {"type": "screenshot", "fullPage": true, "quality": 85}

2) Scrape with Actions (Page Interaction)

cat <<'JSON' | node scripts/firecrawl-api.cjs scrape
{
  "url": "https://example.com",
  "formats": ["markdown"],
  "actions": [
    {"type": "wait", "milliseconds": 2000},
    {"type": "click", "selector": "#load-more"},
    {"type": "wait", "milliseconds": 1000},
    {"type": "scroll", "direction": "down", "amount": 500}
  ]
}
JSON