cheerio

cheerio Best Practices

cheerio is your lightweight, server-side jQuery for Node.js, ideal for fast HTML parsing and manipulation. Follow these rules to build efficient, maintainable, and robust scraping solutions.

1. Code Organization & Structure

Always modularize your scraping logic. Separate HTTP requests, HTML parsing, and data persistence. Use async/await for all asynchronous operations.

✅ GOOD: Modular & Asynchronous Structure

// index.js
import * as cheerio from 'cheerio';
import axios from 'axios';
import { writeFile } from 'fs/promises';

async function fetchAndParse(url) {
  try {
    const { data: html } = await axios.get(url, {
      headers: { 'User-Agent': 'YourScraper/1.0' }
    });
    const $ = cheerio.load(html);

    const products = [];
    $('.product-item').each((i, el) => {
      const $el = $(el);
      const name = $el.find('.product-name').text().trim();
      const price = $el.find('.product-price').text().trim();
      const imageUrl = $el.find('img').attr('src');
      
      // Always check for existence before pushing
      if (name) { 
        products.push({ name, price, imageUrl });
      }
    });
    return products;
  } catch (error) {
    console.error(`Error scraping ${url}:`, error.message);
    throw error;
  }
}

async function main() {
  const targetUrl = 'https://www.example.com/products';
  const scrapedData = await fetchAndParse(targetUrl);
  await writeFile('products.json', JSON.stringify(scrapedData, null, 2), 'utf8');
  console.log('Scraping complete. Data saved to products.json');
}

main();

2. Common Patterns & Anti-patterns

2.1. Selector Efficiency

Keep selectors specific and scoped. Avoid broad selectors that force cheerio to traverse large portions of the DOM. Chain .find() for nested elements.

// ❌ BAD: Overly broad, inefficient
const allLinks = $('a').map((i, el) => $(el).attr('href')).get();

// ✅ GOOD: Scoped to relevant section
const productLinks = $('#product-grid a.product-link').map((i, el) => $(el).attr('href')).get();

// ✅ GOOD: Chain .find() for precision
const productName = $('.product-card').first().find('h2.title').text().trim();