scraping-reddit

Skill: Reddit SaaS Opportunity Hunter

Production-quality Reddit scraping toolkit for discovering unaddressed user frustrations and emerging market trends for software development.

Overview

This skill provides a comprehensive framework for Reddit-based SaaS opportunity discovery through three distinct layers:

1. Navigation - Where to look (subreddit discovery, wiki scanning, contributor analysis)
2. Extraction - What to pull (high-intent patterns, sentiment scoring, velocity tracking)
3. Synthesis - How to interpret (trend persistence, cross-pollination, opportunity scoring)

Prerequisites

Required Credentials

The skill uses the same credential hierarchy as saas-idea-finder:

1. Environment variables (highest priority)
2. Optional vault/secret store (for service users like openclaw)
3. Shell startup file such as ~/.bashrc (fallback)

Environment variables:

export REDDIT_PROXY_URL="http://user:pass@proxy:8080"  # Residential proxy (recommended)
export REDDIT_CLIENT_ID="your_reddit_app_id"           # OAuth2 (optional, for higher rate limits)
export REDDIT_CLIENT_SECRET="your_reddit_app_secret"   # OAuth2 (optional)

Optional vault/secret store setup:

vault kv put secret/skills-apis/reddit/proxy_url value="http://user:pass@proxy:8080"
vault kv put secret/skills-apis/reddit/client_id value="your_client_id"
vault kv put secret/skills-apis/reddit/client_secret value="your_client_secret"

Rate Limits

• Unauthenticated: ~10 requests/minute (6 second delays)
• OAuth2 Authenticated: 100 requests/minute (0.6 second delays)
• With Residential Proxy: More reliable access, less blocking

Quick Start

Basic Usage

import asyncio
from scripts import RedditScraper, OpportunityAnalyzer

async def main():
    # Initialize scraper (auto-loads credentials)
    scraper = RedditScraper()
    
    # Fetch posts from a subreddit
    posts = await scraper.get_subreddit_posts("startups", limit=50)
    
    # Analyze for opportunities
    analyzer = OpportunityAnalyzer()
    for post in posts:
        analysis = analyzer.analyze_post(post)
        if analysis["intent_signals"]:
            print(f"💡 Opportunity: {post.title}")
            print(f"   Problem: {analysis['problem_statement']}")
    
    await scraper.close()

asyncio.run(main())