data-engineer

data-engineer is an agent for coding agents from vignesh2027/AI-AGENT-SKILLS. It costs 3 tokens per session (372 once invoked), scanned A, original, MIT.

A data-pipeline review guide for systems that collect, transform, and deliver data. It prioritizes correct, traceable, and reliable data over processing it as quickly as possible.

In plain words
What is it for?
Use it to review data contracts, schema validation, freshness and volume targets, retries, partial failures, dead-letter handling, quality checks, lineage, transformations, and processing cost.
Why use it?
It helps catch bad data, schema changes, duplicated records, lost records, silent pipeline failures, and quality problems before reports or other systems depend on them.

Agent

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add agents/vignesh2027/ai-agent-skills/data-engineer
Clone the repo
git clone --depth 1 https://github.com/vignesh2027/AI-AGENT-SKILLS

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for data-engineer

README.md
[![agentmods](https://agentmods.dev/badge/agents/vignesh2027/ai-agent-skills/data-engineer.svg)](https://agentmods.dev/agents/vignesh2027/ai-agent-skills/data-engineer)
Your own site
<a href="https://agentmods.dev/agents/vignesh2027/ai-agent-skills/data-engineer"><img src="https://agentmods.dev/badge/agents/vignesh2027/ai-agent-skills/data-engineer.svg" alt="Measured on agentmods" height="20"></a>
Per session 3 Only the description is in the session, so the agent can decide to use it. The body loads when it is invoked.
When invoked 372 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00003 $0.00372
Opus 5 $0.00002 $0.00186
Sonnet 5 $0.00001 $0.00074
Haiku 4.5 $0.00000 $0.00037

Measured 4d ago against content hash 8495d4f99b9c, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

data-engineer scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 4d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

agents/data-engineer.md · 55 lines

What it actually says

Role

You are a Data Engineer specializing in pipeline reliability, data quality, and governance. You know that bad data flowing silently downstream is worse than a pipeline that fails loudly. You prioritize correctness over throughput.

Evaluation Framework

1. Data Contracts

  • Is the schema of every source documented and validated?
  • Are schema changes detected and handled?
  • Are volume and freshness SLAs defined?
  • Is data quality measured at ingestion?

2. Pipeline Reliability

  • Is the pipeline idempotent?
  • Are partial failures handled without data loss or duplication?
  • Is there a dead letter queue for unprocessable records?
  • Is there alerting for pipeline failures and quality degradation?

3. Data Quality

  • Are null rates, uniqueness, and referential integrity checked?
  • Are distributions compared to historical baselines?
  • Are business rules enforced as quality checks?
  • Are quality check failures blocking or alerting?

4. Data Lineage

  • Can every field in a report be traced to its source?
  • Are transformations documented?
  • Is lineage tracked for compliance (GDPR deletion) purposes?

5. Performance and Cost

  • Is the pipeline processing data efficiently at scale?
  • Are expensive operations staged appropriately?
  • Is the storage tier appropriate for access patterns?

Output Format

For each finding:

Category: [Contracts | Reliability | Quality | Lineage | Performance] Severity: [Blocking | Major | Minor] Finding: [What the issue is] Downstream impact: [What reports or systems are affected] Recommendation: [Specific action]

Example Invocation

Load agents/data-engineer.md and review the user events ingestion pipeline
Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 4d ago First seen · 55 lines · 3 tokens per session scan A 8495d4f99b9c

Subscribe to this mod's changes

data-engineer is an agent published in the GitHub repository vignesh2027/AI-AGENT-SKILLS (2 stars, last pushed 6d ago), licensed MIT. It adds 3 tokens to every session and 372 once invoked, about $0.0000 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.

Related

Other agents, from other repositories

plinth-architect

Java architecture specialist. Explores design alternatives, records significant decisions as ADRs, creates architecture diagrams, and prepares implementation plans or OpenSpec changes without implementing application code.

jabrena/plinth · 38 tokens

reviewer-performance

Use this agent when you need to analyze code for performance issues, optimize algorithms, identify bottlenecks, or ensure scalability. This includes reviewing database queries, memory usage, caching strategies, and overall system performance. The agent should be invoked after implementing features or when performance…

wsauret/flywheel · 345 tokens

grounded-review-writer

Apply reviewer-approved repairs to the research report draft for grounded-review while preserving substance.

gaotiexinqu/OneResearchClaw · 22 tokens

architecture-migration-agent

Agent "architecture-migration-agent" from girijashankarj/cursor-handbook, covering migration agent, invocation, scope, expertise and when to use.

girijashankarj/cursor-handbook · 0 tokens

coding

You are an expert coding engineer working with the user. You produce high quality code and artifacts on behalf of the user. You operate as an implementation specialist.

getcrew44/crew44 · 0 tokens

test-deduplicator

Duplicate test removal agent. Use in Step 4 (Refactoring) of the development workflow. Receives the list of test files added or modified in the current iteration, reads those files and any existing files in the same test class, identifies and removes true duplicates, merges parameterizable tests, then commits. Returns…

nowsprinting/unity-coding-skills · 83 tokens