Unknown-333

Unknown-333/awesome-data-engineering-skills

Plugin Claude Code

Plugin marketplace listing 1 plugin: data-engineering-skills.

Unknown-333/awesome-data-engineering-skills

Plugin Claude Code

Curated, portable Agent Skills for data engineers.

Unknown-333/awesome-data-engineering-skills

Skill Claude CodeCodex

Write production-grade Apache Airflow DAGs using the TaskFlow API — idempotent tasks, correct scheduling and catchup, retries/SLAs, connections/variables, and avoiding top-level code. Use when creating or reviewing Airflow DAGs, scheduling pipelines, wiring task dependencies, configuring retries/backfills, or fixing…

Unknown-333/awesome-data-engineering-skills

Skill Claude CodeCodex

Build Dagster pipelines using software-defined assets — asset dependencies, partitions, resources and IO managers, asset checks, and schedules/sensors. Use when creating Dagster assets or jobs, modeling data as assets, adding partitions or backfills, wiring resources/IO managers, or migrating from task-based…

Unknown-333/awesome-data-engineering-skills

Skill Claude CodeCodex

Build well-structured dbt models — staging/intermediate/marts layers, ref() and source(), materializations, and incremental models with the right strategy. Use when creating or refactoring dbt models, choosing table vs view vs incremental, structuring a dbt project, or writing incremental logic.

Unknown-333/awesome-data-engineering-skills

Skill Claude CodeCodex

Build ML feature pipelines and feature stores — point-in-time-correct joins to avoid label leakage, offline/online parity, feature freshness and backfills, and materialization with tools like Feast. Use when engineering features for ML, preventing train/serve skew or data leakage, building a feature store, or…

Unknown-333/awesome-data-engineering-skills

Skill Claude CodeCodex

Design and operate Apache Iceberg tables — partitioning and hidden partitioning, partition/schema evolution, snapshots and time travel, compaction and small-file cleanup, and MERGE/upsert for lakehouse tables on Spark, Flink, Trino, or Snowflake. Use when creating or maintaining Iceberg tables, choosing partitioning…

Unknown-333/awesome-data-engineering-skills

Skill Claude CodeCodex

Build batch and incremental data ingestion (extract-load) pipelines — full vs incremental extraction, change data capture (CDC), watermarks and high-water marks, API pagination and rate limits, and choosing managed EL tools (Fivetran, Airbyte) vs custom code. Use when ingesting data from databases, APIs, files, or…

Unknown-333/awesome-data-engineering-skills

Skill Claude CodeCodex

Build reliable Apache Kafka consumers and producers — consumer groups and partition assignment, offset commit strategy, at-least-once vs exactly-once, idempotent/transactional producers, rebalancing, and dead-letter handling. Use when writing Kafka consumers/producers, configuring offset commits or consumer groups…

Unknown-333/awesome-data-engineering-skills

Skill Claude CodeCodex

Diagnose failing or stuck Apache Airflow pipelines — task failures and retries, scheduler/executor problems, XCom errors, zombie/queued tasks, dependency deadlocks, and pools/concurrency limits. Use when an Airflow task fails or is stuck queued, the scheduler is not running tasks, a DAG will not trigger, or tasks…

Unknown-333/awesome-data-engineering-skills

Skill Claude CodeCodex

Systematically root-cause data pipeline failures and data incidents — job errors, wrong or missing data, duplicates, and freshness misses — by tracing lineage upstream, isolating the failing stage, reconciling against source, and planning a safe fix and backfill. Use when a pipeline fails, numbers look wrong, data is…

Unknown-333/awesome-data-engineering-skills

Skill Claude CodeCodex

Diagnose and fix failing dbt runs — Jinja/compilation errors, ref/dependency and DAG issues, incremental models producing wrong or duplicate rows, full-refresh needs, and state/deferral problems. Use when dbt run or dbt build fails, a model compiles wrong, an incremental model is stale or duplicated, or a CI dbt job…

Unknown-333/awesome-data-engineering-skills

Skill Claude CodeCodex

Plan and run safe data backfills and replays — idempotent reprocessing of historical windows, partition-by-partition execution, isolating backfill compute from production, verifying results, and avoiding double-counting or changed history. Use when backfilling a new or fixed model, reprocessing after a bug, replaying…

Unknown-333/awesome-data-engineering-skills

Skill Claude CodeCodex

Define and enforce data contracts between producers and consumers — explicit schema, semantics, ownership, SLAs, and versioning — to prevent silent upstream changes from breaking downstream pipelines. Use when a producer schema change could break consumers, defining an interface between teams/services and the…

Unknown-333/awesome-data-engineering-skills

Skill Claude CodeCodex

Structure a lakehouse with the medallion architecture — bronze (raw), silver (cleaned/conformed), and gold (business/aggregated) layers — with clear responsibilities, idempotent layer transitions, and where to put quality checks and modeling. Use when organizing a data lakehouse, defining bronze/silver/gold layers…

Unknown-333/awesome-data-engineering-skills

Skill Claude CodeCodex

Document a dbt project — model and column descriptions in schema.yml, source definitions, exposures, doc blocks, and generated docs/lineage. Use when adding descriptions to dbt models, documenting sources or dashboards as exposures, setting up dbt docs, or improving data catalog coverage.

Unknown-333/awesome-data-engineering-skills

Skill Claude CodeCodex

Build reliable Databricks pipelines on the lakehouse — Delta Lake tables, MERGE and time travel, Delta Live Tables / Lakeflow declarative pipelines, Unity Catalog governance, Photon, Auto Loader ingestion, and cluster/job sizing. Use when building Databricks jobs or DLT pipelines, working with Delta tables, ingesting…

Unknown-333/awesome-data-engineering-skills

Skill Claude CodeCodex

Generate realistic synthetic data for testing data pipelines — deterministic seeded fixtures, referential integrity across tables, edge cases (nulls, duplicates, late/out-of-order events), volume for load tests, and privacy-safe stand-ins for production. Use when creating test data for pipeline/dbt tests, seeding dev…

Unknown-333/awesome-data-engineering-skills

Skill Claude CodeCodex

Evolve data schemas safely over time — backward/forward compatibility, additive vs breaking changes, column adds/renames/type changes, and evolution in Avro, Parquet, Iceberg, Delta, and warehouse tables. Use when changing a table or event schema, adding or renaming columns, changing types, or preventing a schema…

Unknown-333/awesome-data-engineering-skills

Skill Claude CodeCodex

Capture database changes with Debezium change data capture — connector setup for Postgres/MySQL/SQL Server, snapshot vs streaming phases, handling inserts/updates/deletes and tombstones, schema changes, and applying the change stream idempotently to a warehouse/lake. Use when setting up CDC, replicating an OLTP…

Unknown-333/awesome-data-engineering-skills

Skill Claude CodeCodex

Set up CI/CD for data pipelines — SQL/dbt linting (SQLFluff), compilation and test gates, dbt Slim CI with state:modified and deferral, environment promotion (dev/staging/prod), and running only changed models on pull requests. Use when adding CI checks to a dbt or SQL project, automating pipeline tests on PRs…

Unknown-333/awesome-data-engineering-skills

Skill Claude CodeCodex

Add data quality checks to pipelines — freshness, volume/row-count anomalies, schema drift, null/uniqueness/referential integrity, and value distributions — using dbt tests, Great Expectations, or Soda, and deciding warn vs block. Use when adding data quality validation, catching bad data before it reaches consumers…

Unknown-333/awesome-data-engineering-skills

Skill Claude CodeCodex

Instrument data pipelines with observability — freshness and volume SLAs, run success/latency metrics, data quality monitors, anomaly detection, lineage, and actionable alerting that avoids fatigue. Use when pipelines fail silently, incidents are found by stakeholders instead of alerts, setting up monitoring/SLAs for…

Unknown-333/awesome-data-engineering-skills

Skill Claude CodeCodex

Capture and use data lineage with OpenLineage and Marquez — emitting run/job/dataset events from Airflow, dbt, and Spark, column-level lineage, and using lineage for impact analysis, debugging, and backfill scoping. Use when setting up data lineage, integrating OpenLineage, tracing what a change breaks downstream, or…