thirdwatch-dev/scraping-skills
Skill Claude CodeCodex
Use when a site blocks your scraper and you need to get past it — 403 Forbidden on the first request, JS challenges, CAPTCHAs, Cloudflare Turnstile, DataDome, Akamai, PerimeterX, "Just a moment...", access denied, IP bans, or an empty/skeleton page where data should be. Covers diagnosing transient failures vs real…
thirdwatch-dev/scraping-skills
Skill Claude CodeCodex
Use when you want to package a web scraper as a deployable, monetizable Apify Actor in Python — set up the directory structure, choose a runtime pattern (HTTP / impit / Playwright / Camoufox), write the input/output/dataset schemas, deploy with the apify CLI, and configure pay-per-event pricing. Triggers on "build an…
thirdwatch-dev/scraping-skills
Skill Claude CodeCodex
Use when you need B2B leads or company data — business listings with contact info, supplier directories, company/people enrichment, KYB/verification, or trade data. Covers Google Maps (local listings), IndiaMart and JustDial (supplier/business directories), LinkedIn (company employees and candidate finder), GST…
thirdwatch-dev/scraping-skills
Skill Claude CodeCodex
Use when you need to scrape e-commerce product data — titles, brands, prices, MRP/list price, discounts, ratings, variants/sizes, availability, images, and product URLs — from online storefronts and marketplaces. Covers Amazon, Flipkart, AliExpress, Myntra, Nykaa, Meesho, Snapdeal, Noon, AJIO, FirstCry, Tata Cliq, and…
thirdwatch-dev/scraping-skills
Skill Claude CodeCodex
Use when you need data from food-delivery platforms — restaurants, menus, item prices, ratings, and delivery times/fees — for menu and price intelligence, restaurant coverage analysis, market analysis, or competitor menu tracking. Covers Swiggy and Zomato (India), Talabat (Middle East), Deliveroo (UK/UAE/EU), and Noon…
thirdwatch-dev/scraping-skills
Skill Claude CodeCodex
Use when scraping the job market — job listings, salaries, company ratings, or candidate/profile sourcing — from LinkedIn, Indeed, Glassdoor, Naukri, Google Jobs, RemoteOK, Wellfound, Monster, ZipRecruiter, Reed, Adzuna, Upwork, CutShort, or AmbitionBox. Covers building a job aggregator, salary benchmarking…
thirdwatch-dev/scraping-skills
Skill Claude CodeCodex
Use when you need property or real-estate listing data — sale and rental listings, asking prices, BHK / bed-bath counts, carpet/built-up area, floor, location and coordinates, and agent/owner details. Covers Rightmove (UK), 99acres, MagicBricks, NoBroker, CommonFloor (India), and Craigslist housing. Triggers on…
thirdwatch-dev/scraping-skills
Skill Claude CodeCodex
Use when scraping reviews, ratings, or reputation data — review text, star ratings, TrustScore, pros/cons, company replies. Covers Trustpilot, G2, Capterra, Yelp, Google Maps, and Shopify review widgets. Triggers on "scrape reviews", "ratings", "brand monitoring", "voice of customer", "VOC", "competitor reviews"…
thirdwatch-dev/scraping-skills
Skill Claude CodeCodex
Use when scraping search engines, SERPs, or SEO data — Google Search organic results, Google News headlines, bulk on-page SEO audits (titles/meta/headings/schema), sitemap and subdomain discovery, or bulk image download from pages. Covers rank tracking, SERP/news monitoring, technical SEO audits at scale, competitive…
thirdwatch-dev/scraping-skills
Skill Claude CodeCodex
Use when you need to scrape social media or content platforms — posts, engagement metrics, profiles, videos, ads, or transcripts — from Twitter/X, Instagram, TikTok, Reddit, LinkedIn, Pinterest, YouTube, Facebook Ad Library, or IMDb. Covers social listening, influencer research, content and trend analysis, ad-creative…
thirdwatch-dev/scraping-skills
Skill Claude CodeCodex
Use when you need travel and hospitality data — hotels, vacation rentals, attractions, or car-sharing listings with their prices, ratings, reviews, availability, star levels, and amenities. Covers Booking.com, Trip.com/Ctrip, TripAdvisor, and Turo. Use for rate and price monitoring, competitive pricing analysis…
thirdwatch-dev/scraping-skills
Skill Claude CodeCodex
Use when starting any web scraping or structured-data-extraction task and you need to decide HOW to get the data — whether to call an API, run a ready-made scraper, or build a custom one. Covers the cost-first technique ladder (plain HTTP → TLS fingerprint spoof → stealth browser → full browser), the build-vs-buy…