Methasit-Pun

Methasit-Pun/data_engineer_claude_skills

Plugin Claude Code

Claude Code skills covering the full data engineering lifecycle, from sourcing to reporting.

Methasit-Pun/data_engineer_claude_skills

Skill Claude CodeCodex

Umbrella skill for running a data project end-to-end through its lifecycle stages — discover sources → profile the data → architect the platform → build the medallion pipeline → refactor the code. Use this whenever the user is kicking off a new data project, asks "where do I start" or "what are the steps", or is…

Methasit-Pun/data_engineer_claude_skills

Skill Claude CodeCodex

Data collection and readiness — the step BEFORE a pipeline exists. Catalog candidate data sources for an objective, rank each by importance/impact, and record where to get it (internal system vs. open/public dataset), how to access it, its refresh cadence, licensing, and readiness blockers. Use this skill whenever a…

Methasit-Pun/data_engineer_claude_skills

Skill Claude CodeCodex

Profile and map raw data BEFORE designing a schema. One-time exploratory analysis to learn the true shape of a dataset — row/column counts, null rates, cardinality, value distributions, ranges, data types, candidate keys, duplicates, referential relationships — then a source-to-target field mapping and an ER diagram.…

Methasit-Pun/data_engineer_claude_skills

Skill Claude CodeCodex

Umbrella skill for shaping data once it has landed — warehouse schema design (star/snowflake/OBT/SCD/grain), SQL for analytics (window functions, CTEs, optimization), dbt model layers/tests/macros/incrementals, and Python transforms (pandas/Polars/PySpark performance). Use this whenever the user is designing tables…

Methasit-Pun/data_engineer_claude_skills

Skill Claude CodeCodex

The most important design decision in a dbt project is how to organize model layers. A clear layer structure means every model has exactly one place it belongs, and anyone reading the project can understand what each model does.

Methasit-Pun/data_engineer_claude_skills

Skill Claude CodeCodex

Pandas, Polars, and PySpark idioms for production data engineering — chunked reads, memory-safe transforms, vectorized operations, type optimization, and performance patterns. Use this skill whenever the user is writing a Python data transformation script and running into memory issues, slow performance, or…

Methasit-Pun/data_engineer_claude_skills

Skill Claude CodeCodex

Data modeling for analytical workloads — star schema, snowflake schema, one big table (OBT), slowly changing dimensions (SCD), normalization tradeoffs, grain definition, and surrogate key strategies. Use this skill whenever the user is designing or reviewing a data warehouse schema, planning a fact/dimension table…

Methasit-Pun/data_engineer_claude_skills

Skill Claude CodeCodex

Best-practice SQL for analytical workloads — window functions, CTEs, query optimization, partitioning strategies, and anti-patterns to avoid. Use this skill whenever the user is writing or reviewing a SQL query that goes beyond a basic SELECT, especially on BigQuery, Snowflake, Redshift, or DuckDB. Trigger on mentions…

Methasit-Pun/data_engineer_claude_skills

Skill Claude CodeCodex

Umbrella skill for where data pipelines run and what they cost — AWS/GCP/Azure infrastructure (S3/GCS/ADLS layout, BigQuery/Redshift/Snowflake selection, IAM, managed-service choice, performance tuning) and cost control (bytes scanned, partition pruning, slot/credit management, storage tiering, budgets and alerts).…

Methasit-Pun/data_engineer_claude_skills

Skill Claude CodeCodex

AWS/GCP/Azure data infrastructure — S3/GCS/ADLS partitioning, BigQuery slot management, Redshift spectrum, Snowflake warehouses, IAM roles for data access, cost optimization, and managed service selection. Use this skill whenever the user is deploying a pipeline to cloud, choosing between managed data services…

Methasit-Pun/data_engineer_claude_skills

Skill Claude CodeCodex

Query cost analysis, partition pruning, slot reservation strategies, storage tiering, and cloud data warehouse cost reduction. Use this skill whenever the cloud data bill is unexpectedly high, a specific query is scanning too much data, the team wants to understand what's driving BigQuery/Snowflake/Redshift costs, or…

Methasit-Pun/data_engineer_claude_skills

Skill Claude CodeCodex

Objective-first data architecture design and decision framework. Establish the target object FIRST (BI, ML, or both), elicit requirements and constraints (cloud vs. on-prem, resource/compute budget, team skills, latency and freshness SLAs, compliance), then design the layered architecture and recommend a model …

Methasit-Pun/data_engineer_claude_skills

Skill Claude CodeCodex

Moving data between systems safely — cutover planning, backfill strategies, dual-write patterns, validation, rollback procedures, and zero-downtime migration techniques. Use this skill whenever the team is migrating from one database or warehouse to another (MySQL → Snowflake, Redshift → BigQuery, on-prem → cloud)…

Methasit-Pun/data_engineer_claude_skills

Skill Claude CodeCodex

Umbrella skill for moving data from source to destination — end-to-end ETL/ELT design, DAG orchestration, real-time streaming, and system-to-system migration. Use this whenever the user is building, scheduling, debugging, or migrating a pipeline and it isn't yet clear which sub-area dominates. This skill ROUTES to the…

Methasit-Pun/data_engineer_claude_skills

Skill Claude CodeCodex

Design a medallion (bronze/silver/gold) ETL architecture interactively, objective-first. List the available data, confirm the objective, then design GOLD first to match the objective and get the user to review it before moving down to silver, then bronze (top-down default) — or bronze-up if the user asks. Asks the…

Methasit-Pun/data_engineer_claude_skills

Skill Claude CodeCodex

Airflow/Prefect/Dagster DAG design — task dependencies, retries, SLAs, backfill strategies, sensors, and failure recovery. Use this skill whenever the user is building or debugging a scheduled pipeline with multiple steps, asking how to handle task failures, setting up retries or alerts, designing a DAG structure…

Methasit-Pun/data_engineer_claude_skills

Skill Claude CodeCodex

Design ETL/ELT pipelines end-to-end — source connectors, extraction strategies, transform logic, load patterns, idempotency, scheduling, and error handling. Use this skill whenever the user is starting a new ingestion job, planning how data moves from a source (REST API, database, file, webhook, message queue) into a…

Methasit-Pun/data_engineer_claude_skills

Skill Claude CodeCodex

Kafka, Flink, Kinesis, and Spark Structured Streaming design — consumer groups, partitioning, exactly-once semantics, lag monitoring, windowing, and late-arriving data. Use this skill whenever the user needs real-time or near-real-time data processing, is redesigning a batch pipeline into streaming, asks about…

Methasit-Pun/data_engineer_claude_skills

Skill Claude CodeCodex

Define and enforce schema contracts between producer and consumer teams — field types, nullability, allowed values, versioning, breaking vs. non-breaking changes, and change detection patterns. Use this skill whenever two teams or services share a dataset and upstream changes keep breaking the downstream silently…

Methasit-Pun/data_engineer_claude_skills

Skill Claude CodeCodex

Data lineage tracking, PII tagging, access control policies, data catalog metadata standards, retention policies, and audit logging for regulatory compliance. Use this skill whenever the company is subject to PDPA, GDPR, HIPAA, or any data privacy regulation, when an audit requires proof of who accesses what data…

Methasit-Pun/data_engineer_claude_skills

Skill Claude CodeCodex

Write systematic data quality checks — validation rules, Great Expectations suites, dbt tests, anomaly detection, null/type/range/referential integrity assertions, and monitoring patterns for production pipelines. Use this skill whenever the user is dealing with bad data in a pipeline, setting up validation before or…

Methasit-Pun/data_engineer_claude_skills

Skill Claude CodeCodex

Umbrella skill for making data correct, trustworthy, and compliant — validation and quality checks (Great Expectations, dbt tests, anomaly/null/range/referential assertions), producer↔consumer schema contracts (versioning, breaking-change detection), and governance (PII tagging, lineage, access control, retention…

Methasit-Pun/data_engineer_claude_skills

Skill Claude CodeCodex

Feature store patterns, training/serving skew prevention, feature pipelines for ML teams, point-in-time correct joins, and bridging data engineering with MLOps conventions. Use this skill whenever an ML team needs feature pipelines, when building a feature store or deciding whether to use one, when there's a…