scraperapi-ruby-sdk

ScraperAPI — Ruby SDK Best Practices

Requires: Ruby >= 2.0, gem install scraperapi (or gem 'scraperapi' in Gemfile), SCRAPERAPI_API_KEY environment variable.

Setup

require "scraper_api"

client = ScraperAPI::Client.new(ENV["SCRAPERAPI_API_KEY"])

Never hardcode the API key. Read it from the environment every time.

Basic Usage

# Simple GET — returns raw HTML string via .raw_body
html = client.get("https://example.com/").raw_body
puts html

# With a single parameter
html = client.get("https://example.com/", render: true).raw_body

# With multiple parameters
html = client.get(
  "https://example.com/",
  render: true,
  country_code: "us"
).raw_body

Parameters are passed as keyword arguments after the URL. .raw_body extracts the HTML string from the response object.

Decision Guide

Situation	Approach
Single URL, synchronous	client.get(url, **params).raw_body
Page loads content via JavaScript	Pass render: true
Site blocks datacenter proxies	Pass premium: true
Toughest anti-bot protection	Pass ultra_premium: true
Multi-step / paginated flow on same domain	Use session_number:
20+ URLs or batch jobs	Use async endpoint via Net::HTTP or Faraday
Supported platform (Amazon, Google, etc.)	Use structured data endpoint directly

Parameter Reference

Rendering

# Render JavaScript before returning HTML
# Use when: page is a React/Vue/Angular SPA, or initial scrape returns empty/partial content
# Cost: +10 credits
html = client.get("https://spa-site.com/", render: true).raw_body

# Wait for a specific DOM element (requires render: true)
html = client.get(
  "https://spa-site.com/",
  render: true,
  wait_for_selector: ".product-list"
).raw_body

Don't add render: true by default — try without it first. It increases cost and latency.

Proxies and Geotargeting

# Route through a country-specific proxy — no extra credit cost
html = client.get("https://example.com/", country_code: "gb").raw_body

# Premium residential/mobile IPs — for sites that block datacenter proxies
# Cost: 10 credits (25 with render: true)
html = client.get("https://hard-site.com/", premium: true).raw_body

# Ultra-premium — for the toughest anti-bot protections
# Cost: 30 credits (75 with render: true)
# Note: incompatible with keep_headers — custom headers are discarded
html = client.get("https://hardest-site.com/", ultra_premium: true).raw_body