build-scrape

Build Scrape Skill

Scrapes a target URL with Playwright headless Chromium, enforces ethical guardrails (robots.txt check + copyright disclaimer + interactive confirmation), sanitises all content with AI-generated placeholder text and dimensions, and produces a clean HTML/CSS directory that feeds directly into the build-visual FSE pipeline — as if the user had exported a Figma or Canva design.

Critical sequencing: This skill runs AFTER build-scaffold and build-mcp complete (WordPress is installed, MCP adapter is active) and BEFORE build-visual Sections 2–6 (CSS token extraction, theme scaffolding, font download, activation, SETUP.md).

This skill expects the following variables to already be set by the calling command:

• SOURCE_URL — the URL to scrape (set by COMMAND.md Section 1 argument parsing)
• BUILD_DIR — absolute path to the build directory (set by build-scaffold Section 2)
• SLUG — build slug (set by COMMAND.md Section 1)
• SITE_TITLE — site title (set by COMMAND.md Section 1)
• PLUGIN_DIR — absolute path to the CoWork plugin directory
• WP — WP-CLI command prefix (set by build-scaffold Section 4)

Section 0: Prerequisite Check

Check that Playwright and robots-parser are installed before proceeding. Create the temp scrape directory and register an EXIT trap for cleanup.

echo "[Build] build-scrape: checking prerequisites..."

# Check Playwright
if ! node -e "require('playwright')" 2>/dev/null; then
  echo ""
  echo "ERROR: Playwright is required for URL clone mode."
  echo "  Install with:"
  echo "    npm install playwright"
  echo "    npx playwright install chromium"
  echo ""
  exit 1
fi

# Check robots-parser
if ! node -e "require('robots-parser')" 2>/dev/null; then
  echo ""
  echo "ERROR: robots-parser is required for URL clone mode."
  echo "  Install with:"
  echo "    npm install robots-parser"
  echo ""
  exit 1
fi

echo "[Build] Prerequisites: playwright OK, robots-parser OK"

# Create temp scrape directory
SCRAPE_DIR="/tmp/scrape_${SLUG}_$$"
mkdir -p "$SCRAPE_DIR"

# EXIT trap: always clean up temp directory (Pitfall 3)
trap "rm -rf '$SCRAPE_DIR'" EXIT

echo "[Build] Scrape temp directory: $SCRAPE_DIR"