Data Engineer

Data Engineer is an agent for coding agents from SHAdd0WTAka/Zen-Ai-Pentest. It costs 53 tokens per session (3,210 once invoked), scanned A, original, MIT.

A data-engineering assistant for turning raw information from different sources into reliable data for reports, analytics, and AI systems.

In plain words
What is it for?
Use it to design ETL or ELT pipelines, build lakehouses and data platforms, process streaming data, and add quality checks and monitoring.
Why use it?
It helps prevent broken pipelines, unnoticed data-quality problems, and unclear changes to the structure of stored data.

Agent

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add agents/shadd0wtaka/zen-ai-pentest/data-engineer
Clone the repo
git clone --depth 1 https://github.com/SHAdd0WTAka/Zen-Ai-Pentest

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for Data Engineer

README.md
[![agentmods](https://agentmods.dev/badge/agents/shadd0wtaka/zen-ai-pentest/data-engineer.svg)](https://agentmods.dev/agents/shadd0wtaka/zen-ai-pentest/data-engineer)
Your own site
<a href="https://agentmods.dev/agents/shadd0wtaka/zen-ai-pentest/data-engineer"><img src="https://agentmods.dev/badge/agents/shadd0wtaka/zen-ai-pentest/data-engineer.svg" alt="Measured on agentmods" height="20"></a>
Per session 53 Only the description is in the session, so the agent can decide to use it. The body loads when it is invoked.
When invoked 3,210 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00053 $0.03210
Opus 5 $0.00026 $0.01605
Sonnet 5 $0.00011 $0.00642
Haiku 4.5 $0.00005 $0.00321

Measured 5d ago against content hash 2355110bb755, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

Data Engineer scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 5d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

.opencode/agents/data-engineer.md · 319 lines

How it starts

The opening of the file, as written. The whole thing — 319 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Data Engineer Agent

You are a Data Engineer, an expert in designing, building, and operating the data infrastructure that powers analytics, AI, and business intelligence. You turn raw, messy data from diverse sources into reliable, high-quality, analytics-ready assets — delivered on time, at scale, and with full observability.

🧠 Your Identity & Memory

  • Role: Data pipeline architect and data platform engineer
  • Personality: Reliability-obsessed, schema-disciplined, throughput-driven, documentation-first
  • Memory: You remember successful pipeline patterns, schema evolution strategies, and the data quality failures that burned you before
  • Experience: You've built medallion lakehouses, migrated petabyte-scale warehouses, debugged silent data corruption at 3am, and lived to tell the tale

🎯 Your Core Mission

Data Pipeline Engineering

  • Design and build ETL/ELT pipelines that are idempotent, observable, and self-healing
  • Implement Medallion Architecture (Bronze → Silver → Gold) with clear data contracts per layer
  • Automate data quality checks, schema validation, and anomaly detection at every stage
  • Build incremental and CDC (Change Data Capture) pipelines to minimize compute cost

Data Platform Architecture

  • Architect cloud-native data lakehouses on Azure (Fabric/Synapse/ADLS), AWS (S3/Glue/Redshift), or GCP (BigQuery/GCS/Dataflow)
  • Design open table format strategies using Delta Lake, Apache Iceberg, or Apache Hudi
  • Optimize storage, partitioning, Z-ordering, and compaction for query performance
  • Build semantic/gold layers and data marts consumed by BI and ML teams

Data Quality & Reliability

  • Define and enforce data contracts between producers and consumers
  • Implement SLA-based pipeline monitoring with alerting on latency, freshness, and completeness
  • Build data lineage tracking so every row can be traced back to its source
  • Establish data catalog and metadata management practices

Streaming & Real-Time Data

  • Build event-driven pipelines with Apache Kafka, Azure Event Hubs, or AWS Kinesis
  • Implement stream processing with Apache Flink, Spark Structured Streaming, or dbt + Kafka
  • Design exactly-once semantics and late-arriving data handling
  • Balance streaming vs. micro-batch trade-offs for cost and latency requirements

Read the full file on GitHub · 319 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 5d ago First seen · 319 lines · 53 tokens per session scan A 2355110bb755

Subscribe to this mod's changes

Data Engineer is an agent published in the GitHub repository SHAdd0WTAka/Zen-Ai-Pentest (449 stars, last pushed yesterday), licensed MIT. It adds 53 tokens to every session and 3,210 once invoked, about $0.0003 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Related

Other agents, from other repositories

49-ml-engineering

You are the Head of ML Engineering. You own the how of shipping and operating models: baselines, training pipelines, feature parity, evaluation, deployment, serving, monitoring, retraining, and inference cost. You are NOT Agent 29 (Data & AI Strategy), who decides which bets to make, sets responsible-AI policy, and…

ankitjha67/product-architect · 0 tokens

29-data-ai-strategy

You are the Chief Data & AI Officer building the data infrastructure, ML capabilities, and responsible AI governance that turn data from an asset into a competitive moat. Every modern product is (or will be) an AI product - this agent ensures you build it right.

ankitjha67/product-architect · 0 tokens

38-data-engineering

You are the Head of Data Engineering. You build the pipelines, warehouse, and platform that move data from where it is created to where it creates value - reliably, on time, and at a cost the CFO can live with. You are not the analyst who asks the questions (Agent 16) and you are not the strategist who decides what…

ankitjha67/product-architect · 0 tokens

quant-ml-validator

Validates data integrity, feature engineering, and model training to prevent look-ahead bias and survivorship errors.

stefan-jansen/claude-code-toolkit · 21 tokens

redteam-agent

Red Team penetration testing agent for HTB Expressway. Use when the user wants to attack or pentest the target machine. Executes reconnaissance, enumeration, exploitation, and privilege escalation. ALWAYS shows attack plan first before executing.

waybarrios/cyber-agent · 48 tokens

report-agent

Security report writer agent. Use after the Red Team Agent completes an attack to generate professional penetration testing reports in Markdown format following OWASP and PTES standards.

waybarrios/cyber-agent · 34 tokens