Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add skills/jamkris/everything-gemini-code/data-scraper-agentnpx skills add Jamkris/everything-gemini-code --skill data-scraper-agentgit clone --depth 1 https://github.com/Jamkris/everything-gemini-codeWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/jamkris/everything-gemini-code/data-scraper-agent)<a href="https://agentmods.dev/skills/jamkris/everything-gemini-code/data-scraper-agent"><img src="https://agentmods.dev/badge/skills/jamkris/everything-gemini-code/data-scraper-agent.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00097 | $0.06080 |
| Opus 5 | $0.00048 | $0.03040 |
| Sonnet 5 | $0.00019 | $0.01216 |
| Haiku 4.5 | $0.00010 | $0.00608 |
Grade A, and why
data-scraper-agent scanned grade A with 1 finding against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Makes network callslowCapability
Not a fault in itself. Listed so you know the mod talks to something, and to what.
resp = requests.get("https://api.example.com/items", headers=HEADERS, timeout=15) This is a copy
86% identical to data-scraper-agent — 31 lines differ, which has more behind it and is treated as the original. This page carries a canonical link to it rather than competing with it.
How it starts
The opening of the file, as written. The whole thing — 765 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Data Scraper Agent
Build a production-ready, AI-powered data collection agent for any public data source. Runs on a schedule, enriches results with a free LLM, stores to a database, and improves over time.
Stack: Python · Gemini Flash (free) · GitHub Actions (free) · Notion / Sheets / Supabase
When to Use
- User wants to scrape or monitor any public website or API
- User says "build a bot that checks...", "monitor X for me", "collect data from..."
- User wants to track jobs, prices, news, repos, sports scores, events, listings
- User asks how to automate data collection without paying for hosting
- User wants an agent that gets smarter over time based on their decisions
Core Concepts
The Three Layers
Every data scraper agent has three layers:
COLLECT → ENRICH → STORE
│ │ │
Scraper AI (LLM) Database
runs on scores/ Notion /
schedule summarises Sheets /
& classifies Supabase
Free Stack
| Layer | Tool | Why |
|---|---|---|
| Scraping | requests + BeautifulSoup |
No cost, covers 80% of public sites |
| JS-rendered sites | playwright (free) |
When HTML scraping fails |
| AI enrichment | Gemini Flash via REST API | 500 req/day, 1M tokens/day — free |
| Storage | Notion API | Free tier, great UI for review |
| Schedule | GitHub Actions cron | Free for public repos |
| Learning | JSON feedback file in repo | Zero infra, persists in git |
AI Model Fallback Chain
Build agents to auto-fallback across Gemini models on quota exhaustion:
gemini-2.0-flash-lite (30 RPM) →
gemini-2.0-flash (15 RPM) →
gemini-2.5-flash (10 RPM) →
gemini-flash-lite-latest (fallback)
Batch API Calls for Efficiency
Never call the LLM once per item. Always batch:
# BAD: 33 API calls for 33 items
for item in items:
result = call_ai(item) # 33 calls → hits rate limit
# GOOD: 7 API calls for 33 items (batch size 5)
for batch in chunks(items, size=5):
results = call_ai(batch) # 7 calls → stays within free tier
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 2d ago First seen · 765 lines · 97 tokens per session scan A 43f17b3995ee
data-scraper-agent is a skill published in the GitHub repository Jamkris/everything-gemini-code (87 stars, last pushed 3mo ago), licensed MIT. It adds 97 tokens to every session and 6,080 once invoked, about $0.0005 per session on Opus 5. A static security scan graded it A with 1 finding (makes network calls). It is 86% identical to data-scraper-agent, differing in 31 lines, and is treated as a copy.
Other skills, from other repositories
gemini-interactions-api
Use this skill when writing code that calls the Gemini API for text generation, multi-turn chat, multimodal understanding, image generation, streaming responses, background research tasks, function calling, structured output, or migrating from the old generateContent API. This skill covers the Interactions API, the…
help
Overview of all available Pinecone skills and what a user needs to get started. Invoke when a user asks what skills are available, how to get started with Pinecone, or what they need to set up before using any Pinecone skill.
assistant
Create, manage, and chat with Pinecone Assistants for document Q&A with citations. Handles all assistant operations - create, upload, sync, chat, context retrieval, and list. Recognizes natural language like "create an assistant from my docs", "ask my assistant about X", or "upload my docs to Pinecone".
image-prompt
막연한 요청을 gpt-image-2(Codex $imagegen) 완성 프롬프트로 컴파일하는 스킬. 검증된 규칙 — 티어드 네거티브(기본 전부 긍정형+극소수 화이트리스트 2종), 앞 브래킷 금지·끝 AR 토큰만, 장비는 결과로 환원, HEX 명시, 1행=1컷, 사이즈락 6종, C1C12 플레이북, 화보 Format B(플랫 콤마형), 시네마틱 키아트(C11·그림자서사 shadownarrative), 프레젠테이션/슬라이드 덱(C12), 룩 프리셋 9종(홍대 인디 L9 포함), 홍보판촉물 그래픽 문법 P1P8(타이포-마스크·타이포-환경·오클루전·컬러블로킹 캠페인·메타…
learning-adaptation
This skill should be used when the user wants to "agents that improve over time", "self-improving agents", "reinforcement learning", "few-shot learning", "online learning", "agent fine-tuning", "adapt from experience", "learn from feedback", "update agent behavior", "self-modifying agents", "RLHF", "DPO", "agent that…
rag
This skill should be used when the user wants to "retrieval augmented generation", "RAG", "ground agent in documents", "knowledge base search", "vector search for agents", "semantic document retrieval", "augment LLM with external knowledge", "document QA", "knowledge grounding", "enterprise knowledge agent", "PDF…