Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add Caravaca-Labs/puzzletide-cli --skill puzzletide-agent-evalsgit clone --depth 1 https://github.com/Caravaca-Labs/puzzletide-cliWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/caravaca-labs/puzzletide-cli/puzzletide-agent-evals)<a href="https://agentmods.dev/skills/caravaca-labs/puzzletide-cli/puzzletide-agent-evals"><img src="https://agentmods.dev/badge/skills/caravaca-labs/puzzletide-cli/puzzletide-agent-evals/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/caravaca-labs/puzzletide-cli/puzzletide-agent-evals"><img src="https://agentmods.dev/badge/skills/caravaca-labs/puzzletide-cli/puzzletide-agent-evals.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00065 | $0.00821 |
| Opus 5.5 | $0.00026 | $0.00328 |
| Sonnet 5.5 | $0.00013 | $0.00164 |
| Haiku 4.5 | $0.00006 | $0.00082 |
Grade A, and why
puzzletide-agent-evals scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 1mo ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 80 lines — stays where its author put it; the contents beside it link to each section on GitHub.
PuzzleTide Agent Evals
Generate reproducible, objectively gradable puzzle tasks for testing models and agents with the local PuzzleTide CLI.
Why puzzles: they are verifiable by construction. A sudoku answer either satisfies the rules and preserves the givens or it doesn't; a word search answer either spells the word along a straight line in the grid or it doesn't. Grading needs no LLM judge and no trusted answer key.
Prefer the local CLI. Check availability in this order:
ptide --version
puzzletide --version
npx puzzletide --version
If none of those work, ask the user before installing (npm install -g puzzletide).
Generate a task set
ptide eval generate --type sudoku --n 20 --difficulty hard --seed 1 --out tasks.json
ptide eval generate --type wordsearch --n 10 --difficulty medium --seed 1 --out tasks.json
The tuple (type, difficulty, n, seed) fully determines the task set, so it names a reproducible benchmark — same command, same tasks, on any machine.
Each task has id, instructions, and the puzzle payload:
- sudoku:
puzzle(81 chars,.= empty). Expected answer: completed 81-char string. - wordsearch:
grid(array of row strings) andwords. Expected answer: JSON array of{word, startRow, startCol, endRow, endCol}(0-indexed).
Run the subject model
Send each task's instructions + payload to the model under test and collect
answers as a JSON array of {id, answer}.
Grade
ptide eval check --tasks tasks.json --answers answers.json --json
Returns per-task pass/fail with reasons and a summary score. Grading is deterministic and local.
Links
- Source and docs: GitHub · npm · CLI reference
- From the makers of puzzletide.com — free word search, crossword, and sudoku puzzles
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 1mo ago First seen · 80 lines · 65 tokens per session scan A 59958a6d0924
puzzletide-agent-evals is a skill published in the GitHub repository Caravaca-Labs/puzzletide-cli (0 stars, last pushed 3mo ago), licensed MIT. It adds 65 tokens to every session and 821 once invoked, about $0.0003 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
Other skills, from other repositories
test-failure-explorer
Import and query an existing Vitest 3.2 JSON report to explore test failures, file-level errors and original diagnostic evidence.
cw-gates
Use before claiming any Codewhale change is done, green, or ready to land: the focused-to-broad verification ladder, the budget checks CI enforces, and the rules for what counts as a passing test.
llm-as-judge-evaluation
Evaluate LLM outputs using frontier models as judges. Use for pairwise model comparison, quality scoring with custom rubrics, and automated evaluation pipelines. Covers position bias mitigation, statistical significance, and generating preference data for DPO/RLHF.
skillshare-implement-feature
Implement a feature from a spec file or description using TDD workflow. Use this skill whenever the user asks to: add a new CLI command, implement a feature from a spec, build new functionality, add a flag, create a new internal package, or write Go code for skillshare. This skill enforces test-first development…
dd-triage-flaky-test
Load when investigating a specific flaky test. Gets history, failure pattern, and category, then recommends fix, quarantine, or escalate.
atmos-validation
Validate Atmos projects, components, arbitrary JSON Schema inputs, EditorConfig, and GitHub Actions; use affected-file selection and native CI annotations.