dedupe-setup

A setup command for finding duplicate sections in Markdown files. It turns supplied files or directories into a checked, structured list of text blocks for later comparison.

In plain words
What is it for?
Use it as the first step when reviewing documentation for repeated content, before running duplicate analysis.
Why use it?
It catches missing paths, resolves duplicate file references, respects .gitignore by default, and applies safety checks before analysis begins.

Command

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add commands/axiomantic/spellbook/dedupe-setup
Clone the repo
git clone --depth 1 https://github.com/axiomantic/spellbook
Per session 42 Only the description is in the session, so the agent can decide to use it. The body loads when it is invoked.
When invoked 2,442 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00042 $0.02442
Opus 5 $0.00021 $0.01221
Sonnet 5 $0.00008 $0.00488
Haiku 4.5 $0.00004 $0.00244

Measured 2d ago against content hash dc6ec6a48b72, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

dedupe-setup scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

commands/dedupe-setup.md · 276 lines

How it starts

The opening of the file, as written. The whole thing — 276 lines — stays where its author put it; the contents beside it link to each section on GitHub.

MISSION

Phase 1 of the dedupe skill: turn an operator-supplied seed (file paths or directories of markdown) into a deterministic, reviewable blocks manifest that downstream phases consume.

This command is orchestrator-only. No subagents are dispatched here. All work is local filesystem inspection, segmentation, and manifest emission.

Part of the dedupe- command family.* Run before /dedupe-analyze.

Invariant Principles

  1. Refuse on bad input — every seed path must exist. A missing path aborts the run with an explicit error and emits no manifest.
  2. Resolve canonically — symlinks and relative paths are reduced to their canonical absolute form before deduplication.
  3. Respect .gitignore by default — gitignored files are excluded unless the operator passes --include-gitignored.
  4. Segmentation is delegated — block boundaries and the bucket key / finding_id derivation are defined in skills/dedupe/references/segmentation-protocol.md. This command applies that protocol; it does not restate it.
  5. Mechanical safety floor is applied at setup time — the pattern table in skills/dedupe/references/safety-markers.md is matched against every block body, and matching blocks are force-marked inline_mandatory_mechanical=true before any later phase runs.
  6. No Python, no shell scripts — POSIX utilities only (grep, awk, sed, shasum, printf, find, git ls-files), invoked via the harness Bash tool.

Safety-floor invariant: The mechanical safety floor runs as part of setup. Blocks flagged by the safety-marker predicate carry that flag forward through every later phase. The LLM safety screen in analyze may only ADD INLINE-MANDATORY status to a block, never remove mechanically-assigned status.


Invocation

/dedupe-setup <seed-path-or-comma-list> [--include-gitignored]
  • <seed-path-or-comma-list> — one or more paths separated by commas. Each entry may be a single markdown file or a directory. Directories are walked recursively for *.md files.
  • --include-gitignored — opt-in flag; when set, .gitignored files under the seed are included in the manifest.

Read the full file on GitHub · 276 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 2d ago First seen · 276 lines · 42 tokens per session scan A dc6ec6a48b72

Subscribe to this mod's changes

dedupe-setup is a command published in the GitHub repository axiomantic/spellbook (10 stars, last pushed 7d ago), licensed MIT. It adds 42 tokens to every session and 2,442 once invoked, about $0.0002 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.