ds-eval

ds-eval is a skill for Claude Code from Dataslayer-AI/Marketing-skills. It costs 76 tokens per session (767 once invoked), scanned A, original, MIT.

A testing tool for checking whether agent skills are triggered by the right user requests. It compares test phrases with skill descriptions and labels the result as passed, failed, or ambiguous.

In plain words
What is it for?
Running trigger tests from a YAML file, comparing each phrase with available skill descriptions, and identifying incorrect or unclear matches.
Why use it?
It helps find descriptions that trigger the wrong skill or leave two skills competing for the same request.

Skill for Claude Code

Written for Claude Code: allowed-tools in frontmatter. Also seen: model in frontmatter; mentions Claude Code.

Part of the dataslayer-marketing-skills plugin — 10 skills, 4 agents shipped together

Good fit Running trigger tests from a YAML file, comparing each phrase with available skill descriptions, and identifying incorrect or unclear matches.

Compare 6 skills from other repositories ↓
Install with agentmods
npx agentmods add skills/dataslayer-ai/marketing-skills/ds-eval
Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

Any agent
npx skills add Dataslayer-AI/Marketing-skills --skill ds-eval
Clone the repo
git clone --depth 1 https://github.com/Dataslayer-AI/Marketing-skills

Made for: Claude Code.

Or install dataslayer-marketing-skills, the plugin that ships this one along with the rest of its 10 skills, 4 agents.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for ds-eval

README.md
[![agentmods](https://agentmods.dev/badge/skills/dataslayer-ai/marketing-skills/ds-eval/github.svg)](https://agentmods.dev/skills/dataslayer-ai/marketing-skills/ds-eval)
Your own site
<a href="https://agentmods.dev/skills/dataslayer-ai/marketing-skills/ds-eval"><img src="https://agentmods.dev/badge/skills/dataslayer-ai/marketing-skills/ds-eval/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

agentmods 80×15 button for ds-eval

Your own site · 80×15
<a href="https://agentmods.dev/skills/dataslayer-ai/marketing-skills/ds-eval"><img src="https://agentmods.dev/badge/skills/dataslayer-ai/marketing-skills/ds-eval.svg" alt="Reviewed on agentmods" width="80" height="20"></a>
Per session 76 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 767 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00076 $0.00767
Opus 5 $0.00038 $0.00383
Sonnet 5 $0.00015 $0.00153
Haiku 4.5 $0.00008 $0.00077

Measured 10d ago against content hash da894744e4ec, method: parsed. Prices are Anthropic first-party input rates as of 2026-09-09, from the pricing page.

Security

Grade A, and why

ds-eval scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 10d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

skills/ds-eval/SKILL.md · 107 lines

How it starts

The opening of the file, as written. The whole thing — 107 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Triggering accuracy eval (ds-eval)

You are a QA evaluator for Claude Code skill descriptions. Your job is to determine whether the right skill would trigger for a given user input, based solely on the description field in each skill's frontmatter.

Process

Step 1 — Load test cases and descriptions

Read the test file: !cat "${CLAUDE_SKILL_DIR}/eval/triggering-tests.yaml" 2>/dev/null || echo "No test file found."

Read all skill descriptions by loading each SKILL.md frontmatter from the sibling skill directories. Extract only the name and description fields from each.

If the user passed a filter as argument, only run tests for: $ARGUMENTS

Step 2 — Evaluate each test case

For each test case in the YAML file:

  1. Read the input phrase
  2. Compare it against ALL skill descriptions
  3. Determine which skill's description is the best match for that input
  4. Check:
    • Does the best match equal expected_skill? → PASS
    • Does the best match appear in should_not_trigger? → FAIL
    • Is it ambiguous (two descriptions match equally well)? → AMBIGUOUS

Matching criteria — A description "matches" an input when:

  • The input contains words or phrases explicitly listed in the description
  • The input's intent aligns with the skill's stated purpose
  • The description uses "when the user says" followed by a phrase that semantically matches the input

Do NOT match based on:

  • General topic overlap (e.g., "organic" doesn't auto-match all SEO skills)
  • The body of the SKILL.md — only the description field matters for triggering

Step 3 — Report results

Present results in this format:


Triggering eval results — [date]

Summary: X/Y passed | Z failed | W ambiguous


Passes
Input Expected Matched Result
... ... ... PASS
Failures

For each failure, explain:

  • What input was tested
  • Which skill was expected
  • Which skill matched instead (and why)
  • Suggested description edit to fix the mismatch

Read the full file on GitHub · 107 lines

Files

What ships with it

1 file beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 10d ago First seen · 107 lines · 76 tokens per session scan A da894744e4ec

Subscribe to this mod's changes

ds-eval is a skill published in the GitHub repository Dataslayer-AI/Marketing-skills (22 stars, last pushed 5mo ago), licensed MIT. It adds 76 tokens to every session and 767 once invoked, about $0.0004 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Related

Other skills, from other repositories

press-clip

Turn a live article URL into a press clip that looks like the real coverage — the publication's own logo, fonts, photos and layout kept intact, the ads and clutter removed, and (for a roundup) just the client's section. Renders to PDF. You inspect each site and tailor the removal; the bundled script carries no…

elvisun/newsjack · 74 tokens

story-origin-check

Recover the first public timestamp and canonical major coverage for a newsjacking signal, then decide whether newer coverage is the same story, a different story, or a materially new development.

elvisun/newsjack · 40 tokens

coverage-tracker

Run a Google Alerts-style keyword coverage tracker. Uses news-search for recent keyword queries, lets the LLM dedupe and classify real features versus junk, stores decisions in SQLite, and alerts only on new real coverage.

elvisun/newsjack · 47 tokens

prompt-proximity-architecture

Turn an approved measurement charter, ICPs, and buyer jobs into a budget-aware prompt coverage blueprint across proximity bands, aided status, information acts, journey states, roles, locales, evidence grades, partitions, and measurement lanes. Use before prompt wording to define required, optional, and prohibited…

elvisun/newsjack · 67 tokens

prompt-set-qa

Gate a prompt universe for schema and provenance completeness, target or campaign contamination, evidence entailment, naturalness, one-concept clarity, architecture consistency, aided status, answer leakage, and semantic duplicates. Use after realistic prompt generation and before human panel selection.

elvisun/newsjack · 56 tokens

realistic-prompt-generation

Generate natural, controlled prompt variants from a target-blind design brief and prompt architecture while preserving approved jobs, acts, journeys, constraints, roles, locales, proximity bands, and evidence language. Use after architecture design and before contamination or semantic QA.

elvisun/newsjack · 55 tokens