Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add skills/caravaca-labs/puzzletide-cli/puzzletide-agent-evalsnpx skills add Caravaca-Labs/puzzletide-cli --skill puzzletide-agent-evalsgit clone --depth 1 https://github.com/Caravaca-Labs/puzzletide-cliWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/caravaca-labs/puzzletide-cli/puzzletide-agent-evals)<a href="https://agentmods.dev/skills/caravaca-labs/puzzletide-cli/puzzletide-agent-evals"><img src="https://agentmods.dev/badge/skills/caravaca-labs/puzzletide-cli/puzzletide-agent-evals.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00065 | $0.00821 |
| Opus 5 | $0.00032 | $0.00411 |
| Sonnet 5 | $0.00013 | $0.00164 |
| Haiku 4.5 | $0.00006 | $0.00082 |
Grade A, and why
puzzletide-agent-evals scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 5d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 80 lines — stays where its author put it; the contents beside it link to each section on GitHub.
PuzzleTide Agent Evals
Generate reproducible, objectively gradable puzzle tasks for testing models and agents with the local PuzzleTide CLI.
Why puzzles: they are verifiable by construction. A sudoku answer either satisfies the rules and preserves the givens or it doesn't; a word search answer either spells the word along a straight line in the grid or it doesn't. Grading needs no LLM judge and no trusted answer key.
Prefer the local CLI. Check availability in this order:
ptide --version
puzzletide --version
npx puzzletide --version
If none of those work, ask the user before installing (npm install -g puzzletide).
Generate a task set
ptide eval generate --type sudoku --n 20 --difficulty hard --seed 1 --out tasks.json
ptide eval generate --type wordsearch --n 10 --difficulty medium --seed 1 --out tasks.json
The tuple (type, difficulty, n, seed) fully determines the task set, so it names a reproducible benchmark — same command, same tasks, on any machine.
Each task has id, instructions, and the puzzle payload:
- sudoku:
puzzle(81 chars,.= empty). Expected answer: completed 81-char string. - wordsearch:
grid(array of row strings) andwords. Expected answer: JSON array of{word, startRow, startCol, endRow, endCol}(0-indexed).
Run the subject model
Send each task's instructions + payload to the model under test and collect
answers as a JSON array of {id, answer}.
Grade
ptide eval check --tasks tasks.json --answers answers.json --json
Returns per-task pass/fail with reasons and a summary score. Grading is deterministic and local.
Links
- Source and docs: GitHub · npm · CLI reference
- From the makers of puzzletide.com — free word search, crossword, and sudoku puzzles
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 5d ago First seen · 80 lines · 65 tokens per session scan A 59958a6d0924
puzzletide-agent-evals is a skill published in the GitHub repository Caravaca-Labs/puzzletide-cli (0 stars, last pushed 1mo ago), licensed MIT. It adds 65 tokens to every session and 821 once invoked, about $0.0003 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
Other skills, from other repositories
improvement-discovery
Heuristics and process for discovering structural improvements in this package. Load when planning a new improvement round — contains the smell taxonomy, analysis workflow, and prioritization framework distilled from many phases of refactoring.
pi-extension-lifecycle
Reference for the Pi coding agent's turn/tool execution model and extension event lifecycle. Use when designing extension timing (when to flush, notify, or intercept), understanding event sequencing, or reasoning about what the agent sees between turns. Includes the verified lifecycle diagram, event handler…
anthropic
Anthropic Claude Pro/Max OAuth compatibility workflow for this repo. Use when debugging Anthropic OAuth failures, misleading extra-usage errors, Pi request shaping, prompt fingerprinting, or deciding between hook-based fixes and deeper provider overrides.
design-review
Review a module's dependency and structural patterns for code smells. Use when adding a parameter to a shared interface, when a dependency bag grows past 5 fields, or when planning a refactoring that touches wiring between layers.
markdown-conventions
Project-specific markdown rules (one-sentence-per-line, compact tables, sequential numbering) and YAML frontmatter schema for plans/retros. Load when writing or editing markdown — contains rules that differ from standard markdownlint defaults.
fallow
Codebase intelligence via fallow CLI — dead code, duplication, complexity, refactoring targets. Load when investigating unused code, planning refactors, or reviewing fallow output.