Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add skills/estuary/agent-skills/derivation-pythonnpx skills add estuary/agent-skills --skill derivation-pythongit clone --depth 1 https://github.com/estuary/agent-skillsWhat it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5 | $0.00149 | $0.04437 |
| Opus 5 | $0.00075 | $0.02218 |
| Sonnet 5 | $0.00030 | $0.00887 |
| Haiku 4.5 | $0.00015 | $0.00444 |
Grade A, and why
derivation-python scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 361 lines — stays where its author put it; the contents beside it link to each section on GitHub.
derivation-python
Estuary derivation implemented as a Python class with async-generator transforms. Runtime is uv, types come from Pydantic models generated at flowctl generate time, and the whole thing is validated in pyright strict mode at publish.
Prereq: read derivation-basics first for concepts, project layout, stateless-vs-stateful, and the universal workflow.
Hard constraint: Python derivations run on private or BYOC data planes only — never on shared public planes. Reach out to your Estuary account manager if you're interested in a private or BYOC data plane.
Docs: concept page · walkthrough · upstream examples (stateful.flow.py, pipeline.flow.py).
When to reach for Python (vs SQL / TypeScript)
- ML / embeddings / tokenisers —
model2vec,fastembed,transformers,tiktoken - Async I/O — per-document
await httpx.get(...), external APIs with bounded concurrency - Heavy Python ecosystem deps —
pandas,pyarrow, custom parsers - Rich type safety — Pydantic validators; pyright-strict at publish
Reach for other skills when plain filter/transform fits SQL (derivation-filter-transform), the logic is reducible (derivation-aggregate-metrics), or you want stateful logic without a private DP (derivation-stateful-logic, SQLite).
How it works
You declare derive.using.python.module: <file>.flow.py. flowctl generate introspects your collection schemas and emits IDerivation, Document, and Request symbols you import. Your Derivation(IDerivation) class has one async generator method per transform — named as snake_case of the YAML transforms[].name. Each transform method consumes Request.Read<TransformName> (typed Pydantic read.doc) and yields Document(...) values. Dependencies in derive.using.python.dependencies are resolved by uv at publish time and at runtime. Optional state is persisted across restarts via __init__ / start_commit / reset lifecycle methods.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 2d ago First seen · 361 lines · 149 tokens per session scan A c179256d9d3f
derivation-python is a skill published in the GitHub repository estuary/agent-skills (7 stars, last pushed 11d ago), licensed Apache-2.0. It adds 149 tokens to every session and 4,437 once invoked, about $0.0007 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
Other skills, from other repositories
stream-processing-designer
Design a stream processing system for unbounded, continuously arriving data. Use when choosing a message broker (Kafka vs RabbitMQ), implementing change data capture (CDC) from PostgreSQL, MySQL, or MongoDB via Debezium or Maxwell, selecting window types for aggregation (tumbling, hopping, sliding, session), joining…
target-connector
This skill should be used when creating a new target connector for CocoIndex to integrate with external systems. It provides guidance on implementing TargetHandler, TargetActionSink, and related types for declarative target state synchronization with change detection and automatic cleanup.
cocoindex
This skill should be used when building data processing pipelines with CocoIndex, a Python library for incremental data transformation. Use when the task involves processing files/data into databases, creating vector embeddings, building knowledge graphs, ETL workflows, or any data pipeline requiring automatic change…
cocoindex-diagrams
This skill should be used when creating, editing, or reviewing inline SVG diagrams for the CocoIndex docs site (anything under docs/src/content/docs/). It encodes the component-based primitive system under docs/src/components/diagrams/, the palette + shape semantics, the preview-and-verify loop using headless Chrome…
upgrade-examples
This skill should be used when upgrading the cocoindex package version in all example pyproject.toml files. It uses regex-based sed commands to efficiently update version constraints across multiple files at once.
developing-kafka-python-client
Use when the user wants to build a Python Kafka producer or consumer, add Schema Registry to existing Python code, migrate from raw JSON to schema-backed serialization, or scaffold a confluent-kafka-python project for Confluent Cloud, local Docker, or WarpStream. Also use when user wants to optimize Python Kafka…