gmaps-scraper: Instructions file for Codex

AGENTS.md — gmaps-scraper

Guide for AI agents (Claude, Cursor, Copilot, etc.) working on this codebase. Read this first before making any changes.

What This Project Is

A Python library that scrapes Google Maps business data using reverse-engineered internal APIs. No official API key. No browser automation. Pure HTTP with the pb= protobuf URL parameter protocol.

If you're an AI agent and a user asks you to:

• "Find coffee shops in Austin" → use gmaps search "coffee shops in Austin"
• "Comprehensively scrape chiropractors in Atlanta" → use gmaps collect "chiropractors" --location "Atlanta, Georgia" -o atlanta.json
• "Scrape at most 20 business websites for contacts" → add --max-contacts 20
• Use gmaps grid ... --bbox ... only when an advanced caller already has an exact boundary.
• "Get details for this place" → use gmaps place ChIJ... --enrich
• "Modify the scraper" → read the sections below

Before You Start

Quick Health Check

cd gmaps-scraper
python -m pytest tests/ -q          # should show 266 passed
python -c "import gmaps; print('OK')" # should print OK

If tests fail or import breaks, stop and investigate before making changes.

Understand the Architecture (2-minute read)

GMapsClient (client.py)          ← entry point, 3 modes
  ├── CookieSession (_auth/)     ← NID/AEC/SOCS consent flow
  ├── HTTPTransport (transport.py) ← UA rotation, jitter, retries
  ├── SearchAPI (_search.py)     ← Phase 1 search + Phase 2 details
  │     ├── places()             ← /search?tbm=map
  │     ├── place_details()      ← /maps/preview/place
  │     └── grid_search()        ← grid subdivision for area coverage
  └── Parser (rpc/parser.py)     ← 58 fields → 8 grouped JSON objects

Three operating modes:

1. Default: Fast search only (~15 fields, no login)
2. Enrich: GMapsClient(enrich=True) → search + place details (~30 fields)
3. Login: GMapsClient(enrich=True, login_cookies="...") → full data (~40 fields)