data-engineering

A discipline for moving data from source systems into usable, reliable datasets through collection, transformation, scheduling, and quality checks.

In plain words
What is it for?
Use it to design or debug data pipelines, choose scheduled or continuous processing, preserve raw data, add quality checks, and make safe reruns.
Why use it?
It reduces silent errors such as duplicated records, late data, and dashboards showing plausible but wrong numbers.

Skill for Claude CodeCodex

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add skills/cbrock84/headcount/data-engineering
Any agent
npx skills add cbrock84/headcount --skill data-engineering
Clone the repo
git clone --depth 1 https://github.com/cbrock84/headcount

Made for: Claude Code, Codex.

Per session 67 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 607 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00067 $0.00607
Opus 5 $0.00034 $0.00303
Sonnet 5 $0.00013 $0.00121
Haiku 4.5 $0.00007 $0.00061

Measured 2d ago against content hash f13f8f585b12, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

data-engineering scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

plugins/data-analytics/skills/data-engineering/SKILL.md · 60 lines

How it starts

The opening of the file, as written. The whole thing — 60 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Data engineering

Pipelines are production systems whose failures are quiet. A broken service pages someone; a broken pipeline produces plausible numbers that people act on for a week.

This is movement and transformation. Schema and semantics belong to data-analytics:data-modeling, policy and stewardship to data-analytics:data-governance.

Land raw, transform downstream

Keep an immutable copy of source data exactly as received. Transformation logic will be wrong at some point, and raw data is what lets you reprocess rather than re-request from a source that may no longer have it.

Business logic belongs downstream where it is visible and testable, not buried in ingestion. The exception is transformation required for privacy — minimization, pseudonymization, dropping fields you have no basis to hold — which belongs at ingest precisely because raw storage is what the obligation attaches to. See legal-risk:privacy-and-data-protection.

Idempotence is the property that matters

Every pipeline will be re-run: after a failure, after a fix, after a late-arriving correction. A re-run that double-counts is worse than a failure, because it produces a wrong answer silently.

Design for exactly-once effect at the destination — deterministic keys, merges rather than blind appends, partitioned overwrites. Then re-running is safe and recovery stops being frightening.

Late, duplicate and out-of-order data

Real sources deliver all three. Decide explicitly, per pipeline: how late is an event still accepted, what happens to one arriving after its window closed, and how duplicates are identified.

Distinguish event time from processing time and partition on event time. Aggregations built on arrival time silently reassign yesterday's activity to today whenever a delivery is delayed.

Test data, not just code

Unit tests on transformation logic catch the wrong class of failure. Most damage comes from data that is valid but wrong. Assert on the data itself, in the pipeline, and fail loudly:

Read the full file on GitHub · 60 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 2d ago First seen · 60 lines · 67 tokens per session scan A f13f8f585b12

Subscribe to this mod's changes

data-engineering is a skill published in the GitHub repository cbrock84/headcount (578 stars, last pushed 3d ago), licensed MIT. It adds 67 tokens to every session and 607 once invoked, about $0.0003 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Related

Other skills, from other repositories

ideogram-ultra

Build Ideogram 4 (Ideogram Ultra) txt2img and img2img workflows with the local open-weights model, dual conditional/unconditional models with DualModelGuider, Qwen3-VL text encoder, and structured JSON ("compositional deconstruction") prompts for strong text rendering and layout control.

artokun/comfyui-mcp · 68 tokens

panel-node-pack-sync

Keep the ComfyUI sidebar panel node-pack (comfyui-agent-panel) in step with the orchestrator after comfyui-mcp updates. Use this whenever the orchestrator was just updated (installcomfyui (action:"selfupdate"), npm i -g comfyui-mcp, a new version in the ENVIRONMENT line), when a panel/bridge command fails in a way…

artokun/comfyui-mcp · 206 tokens

civitai

Discover Civitai models with the BUILT-IN downloadmodel action:"searchcivitai" and install/generate them locally. Find a checkpoint/LoRA/embedding on Civitai, download it into ComfyUI, and use its trigger words. Optionally pair the official Civitai MCP for community features (images browsing, posting, collections).

artokun/comfyui-mcp · 76 tokens

20-client-intake-brief-global

Use when an agency or freelancer takes on a NEW client and needs everything up front — an 11-section intake covering business overview, audience, current marketing, goals, budget, timeline, brand assets, approval process, and success criteria, written for the client to fill in and return. Trigger on 'client intake'…

minhnv0807/ai-business-skills · 168 tokens

aeo-geo

Strategy module for Answer Engine / Generative Engine Optimization — audits AI visibility, restructures content for citation, runs entity-consistency checks across Knowledge Graph, Wikidata, Wikipedia, Crunchbase, and LinkedIn, and produces JSON-LD schema specs, monitoring frameworks, and a 90-day LLM content…

indranilbanerjee/digital-marketing-pro · 156 tokens

brand-setup

Create or update the brand profile every other skill reads — a quick 5-question or full 17-question interactive setup capturing identity, business model, industry and compliance markets, 4-dimension voice scales, channels, goals, and competitors, saved to /.claude-marketing/brands/{slug}/profile.json via…

indranilbanerjee/digital-marketing-pro · 148 tokens