claims-validator

claims-validator

Validate documentation for unsupported claims, made-up metrics, and unverifiable statements.

Triggers

Alternate expressions and non-obvious activations (primary phrases are matched automatically from the skill description):

• "fact-check this" → claim validation
• "verify [claim]" → specific claim check

Purpose

This skill identifies statements that make claims without evidence, including:

• Performance metrics without benchmarks or data
• Time/cost estimates without basis
• Percentage claims without citation
• Comparative statements without baselines
• Features described as implemented that don't exist
• Marketing superlatives presented as facts

Behavior

When triggered, this skill:

1. Scans for metric claims:

   • Percentage improvements ("40% faster", "reduces by 60%")
   • Time estimates ("saves 2-3 hours", "in minutes not hours")
   • Cost projections ("$50-150/month", "ROI of 3x")
   • Performance numbers ("99x faster", "sub-millisecond")

2. Identifies unsupported comparatives:

   • "faster than", "better than", "more efficient"
   • "best", "leading", "revolutionary", "game-changing"
   • "comprehensive", "complete", "full-featured"

3. Checks for feature claims:

   • Commands or flags mentioned that don't exist in codebase
   • Features described in present tense that aren't implemented
   • Integration claims without actual integration code

4. Validates citations:

   • Claims that reference data should have sources
   • Benchmarks should link to methodology
   • Statistics should be reproducible

5. Generates report:

   • List each claim found
   • Classification (metric, comparative, feature, cost)
   • Recommendation (remove, add citation, verify, rephrase)

Claim Categories

Metrics Without Data

# Flagged
"Time Saved: 92-96% (9-15 hours → 45-60 minutes)"
"99x faster routing"
"45x cache speedup"

# Problem
No benchmark data, methodology, or reproducible test

# Fix
Remove claim, or add: "Based on [benchmark/test], measured [how]"