Apify MCP Server connects AI agents to Apify's collection of web scrapers, crawlers, and automation tools through the Model Context Protocol, allowing them to extract data from websites, search engines, social media, maps, and online stores. It is for users who want agents to collect web data through compatible clients such as Claude Code, Claude.ai, Cursor, or VS Code. The catalogue entries configure or extend agent use of this MCP server.
Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add skills/apify/apify-mcp-server/creating-mcp-agent-evalsnpx skills add apify/apify-mcp-server --skill creating-mcp-agent-evalsgit clone --depth 1 https://github.com/apify/apify-mcp-serverWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/apify/apify-mcp-server/creating-mcp-agent-evals)<a href="https://agentmods.dev/skills/apify/apify-mcp-server/creating-mcp-agent-evals"><img src="https://agentmods.dev/badge/skills/apify/apify-mcp-server/creating-mcp-agent-evals.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00071 | $0.01854 |
| Opus 5 | $0.00036 | $0.00927 |
| Sonnet 5 | $0.00014 | $0.00371 |
| Haiku 4.5 | $0.00007 | $0.00185 |
Grade A, and why
creating-mcp-agent-evals scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured today.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 69 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Creating MCP agent evals for an MCP tool family
Overview
Build a small, calibrated Langfuse eval suite for one tool family (tasks, storage, runs, …), then use its failures to fix the tools. Core principle: evals are designed from user intent, never from tool descriptions — the eval defines what should work; descriptions get fixed afterward to make naive agents pass it.
Commands, item shapes, probe patterns, and sweep queries: reference.md.
The flow
- Inventory the tools — every tool and every argument group needs at least one case (the coverage matrix at the end proves it).
- Probe the platform first. Before writing any case that depends on API behavior (required fields, uniqueness rules, limits, error messages), verify it with a throwaway
tsxscript against the real API. Never write a case on an assumed contract — that's how you get input values the schema rejects. - Two datasets, never one:
<family>-evals(proper suite, zero tool errors tolerated) and<family>-evals-errors(cases that provoke errors on purpose: collisions, not-found, requirement discovery). Mixing them masks real failures. - Write cases in waves: 2–3 easy (single tool, explicit input) → 1–2 medium (cross-tool chains, run options) → 2–3 hard (vague user language, error recovery, collisions). Run and review each wave before writing the next.
- Calibrate on the strongest model first (Opus). A failure there is a case defect or a product gap — never a description problem. Only a calibrated suite (strong model 100%) can attribute weaker-model failures to descriptions.
- Ladder down (Sonnet → Haiku). Passes-on-Opus-fails-on-Haiku = the tool description or output doesn't carry a naive agent. That's the signal you built the suite for.
- Fix tools via outputs before descriptions. A steering sentence in the tool's response summary/nextStep reaches every agent on every call; description text gets skimmed. Both output nudges that fixed Haiku failures in the original build were response-text changes.
What ships with it
1 file beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- today First seen · 69 lines · 71 tokens per session scan A 076245cea666
creating-mcp-agent-evals is a skill published in the GitHub repository apify/apify-mcp-server (6,088 stars, last pushed today), licensed MIT. It adds 71 tokens to every session and 1,854 once invoked, about $0.0004 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-05.
Other skills, from other repositories
daily-briefing
Start your day with a prioritized CRM briefing — works for any monday CRM setup: pipeline, contacts, accounts, leads, or no board at all. Use when someone says 'morning briefing', 'daily brief', 'what's on my plate today', 'catch me up on my deals', 'what needs attention', 'what needs my attention', 'anything urgent…
data-cleanup
Check your CRM board's health and clean up messy data — run a scored health check (missing fields, stalled deals, abandoned columns, automation gaps) as a report, or fix data in bulk (phone formats, missing emails, country codes, unowned deals, stale dates). Use when someone says "clean my CRM", "fix my data"…
meeting-to-deal
Turn meeting transcripts into deal updates — key points, commitments, and next steps posted to matching CRM deals automatically. Auto-creates contacts for new attendees. Use when someone says "log my meetings to deals", "update CRM from calls", "what did I commit to in meetings", "sync notetaker", "log this meeting"…
workspace-builder
Build a CRM workspace from scratch — describe your business and get boards, columns, and pipeline stages created in monday. Use when someone says "set up my CRM", "build me a CRM", "create CRM boards for me", "I need a new sales pipeline", "help me get started in monday", "I don't know how to set up my pipeline", or…
forecast
Build a forecast dashboard in monday — committed, best-case, and pipeline views by close month, rendered as real dashboard widgets. Use when someone says "build me a forecast", "show me Q2 pipeline", "Salesforce-style forecast", "forecast dashboard", "commit vs best-case", "what's our number this quarter", "how are we…
run-sequence
Manage CRM sequences end-to-end — list, create, enroll contacts, activate/deactivate, duplicate, and track performance. Use when someone says "what sequences do I have", "create a welcome sequence", "enroll this contact in a sequence", "add these leads to my nurture sequence", "deactivate the cold outreach sequence"…