agentsop-idempotent-ingestion

agentsop-idempotent-ingestion is a skill for Claude Code from agentsope/SkillAlchemy. It costs 205 tokens per session (6,943 once invoked), scanned A, original, MIT.

A set of rules for making repeated document imports produce the correct knowledge base. It uses document changes to decide whether to add, update, skip, or remove records; RAG is a system that retrieves relevant documents for generated answers.

In plain words
What is it for?
Building or reviewing scheduled re-indexing, incremental updates, continuous-integration imports, and other ingestion pipelines that run over changing document collections.
Why use it?
It prevents repeated imports from creating duplicates or leaving deleted documents available in search results.

Skill for Claude Code

Written for Claude Code: when-to-use in frontmatter.

Good fit Building or reviewing scheduled re-indexing, incremental updates, continuous-integration imports, and other ingestion pipelines that run over changing document collections.

Compare 6 skills from other repositories ↓
Install with agentmods
npx agentmods add skills/agentsope/skillalchemy/agentsop-idempotent-ingestion
Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

Any agent
npx skills add agentsope/SkillAlchemy --skill agentsop-idempotent-ingestion
Clone the repo
git clone --depth 1 https://github.com/agentsope/SkillAlchemy

Made for: Claude Code.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for agentsop-idempotent-ingestion

README.md
[![agentmods](https://agentmods.dev/badge/skills/agentsope/skillalchemy/agentsop-idempotent-ingestion/github.svg)](https://agentmods.dev/skills/agentsope/skillalchemy/agentsop-idempotent-ingestion)
Your own site
<a href="https://agentmods.dev/skills/agentsope/skillalchemy/agentsop-idempotent-ingestion"><img src="https://agentmods.dev/badge/skills/agentsope/skillalchemy/agentsop-idempotent-ingestion/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

agentmods 80×15 button for agentsop-idempotent-ingestion

Your own site · 80×15
<a href="https://agentmods.dev/skills/agentsope/skillalchemy/agentsop-idempotent-ingestion"><img src="https://agentmods.dev/badge/skills/agentsope/skillalchemy/agentsop-idempotent-ingestion.svg" alt="Reviewed on agentmods" width="80" height="20"></a>
Per session 205 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 6,943 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe. Third-party audits
  • NVIDIA SkillSpector pass 7 Sept 2026
How audits are shown
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00205 $0.06943
Opus 5 $0.00102 $0.03472
Sonnet 5 $0.00041 $0.01389
Haiku 4.5 $0.00020 $0.00694

Measured 11d ago against content hash b7dc283c1470, method: parsed. Prices are Anthropic first-party input rates as of 2026-09-10, from the pricing page.

Security

Grade A, and why

agentsop-idempotent-ingestion scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 11d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

skills/agentsop-idempotent-ingestion/SKILL.md · 540 lines

How it starts

The opening of the file, as written. The whole thing — 540 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Idempotent Ingestion · Re-Ingest-Correctness SOP

Third-person operating model for a coder agent that owns ingestion correctness across repeated runs. The audience is the LLM agent writing or reviewing the pipeline code — not the end user.

One sentence: The interesting run is the second one. A correct pipeline hashes each document, looks the hash up in a docstore, and inserts / updates / skips accordingly — so re-running over unchanged docs changes nothing.

This skill is an ENHANCE overlay over [[llamaindex]]. The base skill names IngestionPipeline with docstore + UPSERTS_AND_DELETE as a hardening must-have (OP-08, anti-pattern A10) but stops at "use it". This overlay surfaces the correctness contract — what idempotency actually means, how the hash decides the action, and how it generalises beyond LlamaIndex.


1. 何时激活 (Activation Rules)

Activate this skill whenever any of the following holds:

  1. The codebase contains an ingestion call (VectorStoreIndex.from_documents, pipeline.run(documents=...), vectorstore.add_documents, a custom embed
    • upsert loop) and that call will execute more than once over a corpus that can change between runs.
  2. The user mentions any of: re-ingest, re-index, scheduled / nightly index refresh, incremental document updates, docstore, doc hash, upsert, RecordManager, "keep the index in sync with the source".
  3. A bug report says "I see the same answer chunk twice", "retrieval returns duplicates", "deleted a file but it still shows up in answers", "the index keeps growing every night even though nothing changed".
  4. PR review: new ingestion code that builds the index with no docstore / no hash-keyed dedup and the corpus is live (LlamaIndex base skill A10).
  5. A production RAG is about to ship and the ingestion side has only ever been tested as a cold first run.

Do not activate when:

  • The index is built once from a static corpus and never refreshed — there is no second run to make idempotent.
  • The corpus is tiny and rebuilding from scratch each run is measurably cheaper than maintaining a docstore (rare; verify, don't assume).
  • The task is purely query-time (retrieval, rerank, synthesis) with no ingestion path in scope.

Read the full file on GitHub · 540 lines

Files

What ships with it

4 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 11d ago First seen · 540 lines · 205 tokens per session scan A b7dc283c1470

Subscribe to this mod's changes

agentsop-idempotent-ingestion is a skill published in the GitHub repository agentsope/SkillAlchemy (394 stars, last pushed 8d ago), licensed MIT. It adds 205 tokens to every session and 6,943 once invoked, about $0.0010 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Related

Other skills, from other repositories

karpathy-llm-wiki

Use when building or maintaining a personal LLM-powered knowledge base. Triggers: ingesting sources into a wiki, querying wiki knowledge, linting wiki quality, 'add to wiki', 'what do I know about', or any mention of 'LLM wiki' or 'Karpathy wiki'.

Astro-Han/karpathy-llm-wiki · 67 tokens

mixedbread-search

Build and query managed search indexes (Stores) using the Mixedbread Python and TypeScript SDKs. Use when creating knowledge bases, uploading documents, performing semantic or vector search, asking questions over documents, using agentic multi-step retrieval, combining store search with web results, filtering by…

mixedbread-ai/skills · 70 tokens

mxbai-cli

Use the mxbai CLI to manage stores, upload files, search documents, ask questions, and sync directories from the terminal. Use when performing Mixedbread operations via command line, setting up CI/CD pipelines with store sync, or managing API keys.

mixedbread-ai/skills · 55 tokens

vector-db-init

Interactively initializes the Vector DB plugin. Guided discovery asks which folders to index, confirms the manifest, then scaffolds vectorprofiles.json for high-performance In-Process or Native Server connections. Mandatory first step before ingestion or search.

richfrem/agent-plugins-skills · 49 tokens

vector-db-launch

Start the Native Python ChromaDB background server. Use when semantic search returns connection refused on port 8110, or when the user wants to enable concurrent agent read/writes.

richfrem/agent-plugins-skills · 40 tokens

vector-db-cleanup

Removes stale and orphaned chunks from the ChromaDB vector store for files that have been deleted or renamed. Use after files are removed or moved to keep the vector index in sync with the filesystem. user: "Clean up the vector store after I deleted some files" assistant: "I'll use vector-db-cleanup to remove orphaned…

richfrem/agent-plugins-skills · 115 tokens