What is data-engineering-agent-skills AGENTS.md?

Repository instructions for data-engineering agents, including commands for defining, planning, building, validating, reviewing, backfilling, and shipping data work. A backfill is the controlled reprocessing of older data.

Use them when an agent works on data pipelines, models, or jobs, especially Python, Scala, Spark, Flink, or JVM projects. They also cover safe replay, deployment, observation, and rollback.

They give agents a standard starting point and route each request to the relevant workflow, while requiring checks for data quality, ownership, lineage, recovery, and contracts.

How vaquarkhan/data-engineering-agent-skills configures Codex and Open

Reuse

Borrowing it

Nothing to install: this file belongs to vaquarkhan/data-engineering-agent-skills. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.

Copy the file

curl -O https://raw.githubusercontent.com/vaquarkhan/data-engineering-agent-skills/main/AGENTS.md

Clone the repo

git clone --depth 1 https://github.com/vaquarkhan/data-engineering-agent-skills

Made for: Codex, OpenCode.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

README.md

[![agentmods](https://agentmods.dev/badge/instructions/vaquarkhan/data-engineering-agent-skills/agents-md/github.svg)](https://agentmods.dev/instructions/vaquarkhan/data-engineering-agent-skills/agents-md)

Your own site

<a href="https://agentmods.dev/instructions/vaquarkhan/data-engineering-agent-skills/agents-md"><img src="https://agentmods.dev/badge/instructions/vaquarkhan/data-engineering-agent-skills/agents-md/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

Your own site · 80×15

<a href="https://agentmods.dev/instructions/vaquarkhan/data-engineering-agent-skills/agents-md"><img src="https://agentmods.dev/badge/instructions/vaquarkhan/data-engineering-agent-skills/agents-md.svg" alt="Reviewed on agentmods" width="80" height="20"></a>

Per session 1,517 This file is loaded in full into every session.

When invoked 1,517 The same file — it is already loaded in full.

Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe.

Origin original No closer match found in the catalogue.

Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

Model	Per session	Once invoked
Fable 5.1	$0.01517	$0.01517
Opus 5.5	$0.00607	$0.00607
Sonnet 5.5	$0.00303	$0.00303
Haiku 4.5	$0.00152	$0.00152

Measured 2mo ago against content hash c6af8b6d56a5, method: parsed. Prices are Anthropic first-party input rates as of 2026-10-07, from the pricing page.

Security

Grade A, and why

data-engineering-agent-skills AGENTS.md scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2mo ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

AGENTS.md · 110 lines

How it starts

The opening of the file, as written. The whole thing — 110 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Data Engineering Agent Entry Point

Use this file as the generic entry point for agents that support AGENTS.md-style repository instructions.

Start Here

Load skills/using-data-engineering-agent-skills/SKILL.md
Load the platform preset from presets/ that matches the environment
Load only the workflow skills needed for the current task
Pull in references, templates, and examples only when they improve decisions or proof

Lifecycle Commands

/spec -> define contract, SLA, lineage, schema, retention, ownership
/plan -> break pipeline changes into atomic tasks
/build -> implement pipeline, model, or job changes incrementally
/validate -> prove data quality, contract compliance, and reconciliation
/review -> review reliability, cost, governance, and operability
/backfill -> run safe replay and cutover workflows
/ship -> deploy, observe, rollback-safe release

Default Routing

unclear request -> data-specification
approved scope -> pipeline-planning-and-task-breakdown
Python data pipeline implementation -> python-data-engineering-and-pipeline-packaging
Scala Spark, Flink, or JVM data job implementation -> scala-data-engineering-on-jvm-runtimes
Java connector, integration, or metadata service work -> java-data-engineering-and-integration-services
file drops, SFTP, or partner-managed feeds -> file-and-partner-feed-ingestion
Glue Data Catalog or Lake Formation governance -> glue-data-catalog-and-lake-formation-governance
Unity Catalog governance -> unity-catalog-and-lakehouse-governance
Purview or Azure governance -> microsoft-purview-and-azure-data-governance
Dataplex or BigQuery governance -> dataplex-and-bigquery-governance
operational store choice such as MySQL versus NoSQL -> operational-datastore-selection-relational-and-nosql
warehouse/dbt work -> warehouse-and-schema-design + dbt-and-analytics-engineering
Snowflake warehouse-native pipelines -> snowflake-native-pipelines-and-governance
BigQuery or Dataform platform work -> bigquery-and-dataform-platform-engineering
ETL or ELT transformation-boundary work -> etl-elt-and-modernization-strategy
mainframe offload or modernization -> mainframe-modernization-and-data-offload
streaming work -> streaming-and-messaging-systems
Kafka production hardening, DLQs, or schema guardrails -> kafka-resilience-and-schema-evolution
serverless Spark on Lambda or short-lived runtimes -> spark-serverless-reliability-and-state-management
agent observability via MCP (lag, Spark plans, run state) -> mcp-data-observability-integration
lakehouse work -> data-lake-and-zone-architecture + lakehouse-table-format-engineering
release workflow, promotion design, or gated rollout -> data-platform-ci-cd-and-release-management
test data or QA datasets -> test-data-preparation-and-synthetic-data
lower-environment refreshes or masked non-prod data -> lower-environment-data-masking-and-obfuscation
Informatica, Talend, or legacy ETL estates -> enterprise-etl-and-data-integration-modernization
regional or country-specific data obligations -> regional-data-compliance-and-sovereignty
ESG or sustainability reporting -> esg-and-sustainability-regulatory-reporting
quality or release gates -> data-quality-and-contract-testing
data-quality tool selection, rule severity, or quality operating model -> data-quality-platforms-and-rule-management
resiliency testing, failure injection, or failover drills -> data-resiliency-testing-and-failure-injection
disaster recovery or business continuity planning -> data-platform-disaster-recovery-and-business-continuity
reliability issue -> data-observability-and-sla-management + incident-triage-and-pipeline-recovery
platform-team ownership, golden paths, or support-boundary design -> data-platform-operating-model-and-service-ownership
replay or migration work -> safe-backfill-and-replay-orchestration + orchestration-and-backfills + data-migration-and-platform-cutover
regulated data and audit-bound publish paths -> data-security-compliance-and-regulated-data + lineage-pii-and-governance

Read the full file on GitHub · 110 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

2mo ago First seen · 110 lines · 1,517 tokens per session scan A c6af8b6d56a5

Subscribe to this mod's changes

data-engineering-agent-skills AGENTS.md is an instructions file published in the GitHub repository vaquarkhan/data-engineering-agent-skills (62 stars, last pushed 4mo ago), licensed MIT. It adds 1,517 tokens to every session, about $0.0061 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Data and AI agent-skills ai-agents apache-spark claude cursor data-engineering

data-engineering-agent-skills AGENTS.md next to 6 instructions from other repositories
Mod	Type	Stars	Last push	Licence	Scan	Tokens	Works with
data-engineering-agent-skills AGENTS.mdvaquarkhan/data-engineering-agent-skills	Instructions file	62	4mo ago	MIT	A	1,517	Codex, OpenCode
vscode buildNext.instructions.mdmicrosoft/vscode	Instructions file	195k	yesterday	MIT	A	9,719	GitHub Copilot
next.js AGENTS.mdvercel/next.js	Instructions file	143k	today	MIT	A	7,461	Codex, OpenCode
langchain AGENTS.mdlangchain-ai/langchain	Instructions file	149k	yesterday	MIT	A	4,680	Codex, OpenCode
vscode oss-third-party-notices.instructions.mdmicrosoft/vscode	Instructions file	195k	yesterday	MIT	A	5,516	GitHub Copilot
vscode copilot-instructions.mdmicrosoft/vscode	Instructions file	195k	yesterday	MIT	A	3,093	GitHub Copilot
deepseek-harness AGENTS.mddeepseek-ai/deepseek-harness	Instructions file	274k	2d ago	MIT	A	4,313	Codex, OpenCode

data-engineering-agent-skills: Instructions file for Codex