Professional Data Engineering Agent Skills for developing AI Agentic Data Platforms.
This repository also configures its own agents. See what de-agent-skills tells them →
Skill Claude CodeCodex
Compliance readiness for data platforms — SOC2 Type II controls mapping (CC6/CC7/CC8), GDPR data processing requirements (PII inventory/right-to-erasure/data residency), PCI-DSS data platform scope reduction, CIS Benchmarks (Kubernetes/AWS/GCP), audit log completeness (Vault/Kubernetes/cloud trail), data retention and…
Skill Claude CodeCodex
Docker best practices — multi-stage builds (builder/runtime separation), layer cache optimization (dependency install before source copy), minimal base images (distroless/alpine/slim), security hardening (non-root USER, read-only FS, no SUID binaries), .dockerignore patterns, BuildKit secrets for private registries…
Skill Claude CodeCodex
GCP data platform review — BigQuery (dataset IAM/column-level security/partitioned tables/clustering/reservations), GCS data lake (uniform bucket-level access/lifecycle/CMEK), Dataproc vs Dataflow vs Spark on GKE, Pub/Sub streaming, Cloud Composer (managed Airflow), Workload Identity for GKE, VPC Service Controls…
Skill Claude CodeCodex
GitOps deployment review — ArgoCD Application/AppProject CRDs, app-of-apps pattern, sync policies (automated+self-heal+prune), drift detection and remediation, FluxCD HelmRelease/Kustomization, git repository structure for GitOps (app code vs config repo separation), image update automation, progressive delivery (Argo…
Skill Claude CodeCodex
Grafana dashboard review — panel types (timeseries/stat/gauge/table/heatmap), variable templating (datasource/labelvalues/query), dashboard linking and drilldown, data platform dashboards (pipeline overview/DAG health/Kafka throughput/Spark performance), alerting from panels, annotation markers for deployments…
Skill Claude CodeCodex
Kafka cost optimization — tiered storage (S3/GCS remote log offload, local.retention.ms vs retention.ms), topic retention right-sizing (per-topic audit SQL), log compaction economics, partition count right-sizing (over-partitioned cluster detection), consumer group idle cleanup, broker instance right-sizing (disk vs…
Skill Claude CodeCodex
Kafka production platform review — broker configuration (replication factor/min.insync.replicas/rack awareness/KRaft mode), topic design (partition count formula/compaction/retention), consumer group management (lag monitoring/rebalance tuning/cooperative sticky), producer tuning…
Skill Claude CodeCodex
Kubernetes autoscaling review — HPA (CPU/memory/custom/external metrics), stabilizationWindowSeconds anti-flapping, VPA vs HPA decision, Cluster Autoscaler tuning (scale-down delay/utilization threshold), KEDA event-driven autoscaling (Kafka lag/queue depth), spot node optimization, pod disruption budgets, scaling…
Skill Claude CodeCodex
Kubernetes cluster health assessment — node status and pressure conditions (disk/memory/PID), pod failure diagnosis (CrashLoopBackOff/OOMKilled/Pending/Evicted), control plane health (API server/etcd/scheduler/controller-manager), resource quota utilization, scheduling failures (taints/affinity/insufficient…
Skill Claude CodeCodex
Kubernetes cost optimization — idle workload detection (zero-replica deployments, always-pending jobs), overprovisioning analysis (request vs actual CPU/memory), namespace-level cost attribution, node bin-packing via requests tuning, spot instance strategy, LimitRange defaults, right-sizing with VPA recommendations…
Skill Claude CodeCodex
Kubernetes security audit — RBAC review (ClusterRoleBindings/RoleBindings/ServiceAccount permissions), Pod Security Standards (privileged/hostPID/hostNetwork/runAsRoot), NetworkPolicy enforcement, Secret management (encryption at rest, external-secrets), image vulnerability scanning (Trivy/Grype), admission…
Skill Claude CodeCodex
Kubernetes storage review — PVC lifecycle (provisioning/binding/reclaim), StorageClass selection (SSD/HDD/NVMe), dynamic vs static provisioning, StatefulSet volume management, volume performance tuning (ReadWriteMany vs ReadWriteOnce), CSI drivers (AWS EBS/GCP PD/Ceph/Longhorn), PVC resize, snapshot and backup…
Skill Claude CodeCodex
Multi-cloud governance — cloud-agnostic data platform patterns, federated identity (OIDC/SAML between AWS/GCP/Azure), Terraform multi-cloud modules, cross-cloud data replication (S3↔GCS/Azure), unified cost management (FinOps Foundation framework), cloud-agnostic observability (OpenTelemetry), policy enforcement (OPA…
Skill Claude CodeCodex
Network security review for data platforms — Kubernetes NetworkPolicy (default-deny + allow patterns), VPC/subnet design (private subnets for all data services), security group rules audit (0.0.0.0/0 detection), TLS everywhere (Kafka mTLS/Trino HTTPS/DB SSL), service mesh (Istio mTLS), DNS security (private hosted…
Skill Claude CodeCodex
Observability stack review — three pillars (metrics/logs/traces), Prometheus+Grafana+Loki+Tempo (LGTM stack), OpenTelemetry Collector as universal agent, cardinality management, log aggregation patterns (Fluent Bit→Loki), distributed tracing (Jaeger/Tempo), alerting pipeline (Alertmanager routing), SLO definition and…
Skill Claude CodeCodex
OpenTelemetry instrumentation for data platforms — Python auto-instrumentation (opentelemetry-bootstrap), manual span creation for ETL tasks, trace context propagation across Airflow→Spark→dbt, OTLP exporter configuration, W3C TraceContext headers in Kafka messages, resource attributes…
Skill Claude CodeCodex
Prometheus optimization — recording rules (level:metric:operations naming), cardinality explosion diagnosis (tsdb status API), scrape interval tuning, remote write configuration (batch size/queue capacity), Thanos/Mimir for long-term retention, federation vs remote write, AlertManager routing trees…
Skill Claude CodeCodex
RBAC audit for data platforms — Kubernetes RBAC (ClusterRole/Role/Binding audit), AWS IAM permission boundary and policy review (AWS IAM Access Analyzer), GCP IAM audit (principle of least privilege/owner role detection), database RBAC (PostgreSQL/Trino/ClickHouse role hierarchies), Airflow RBAC (role-based menu/DAG…
Skill Claude CodeCodex
Secrets management review — HashiCorp Vault (KV v2/dynamic credentials/lease renewal/audit log), External Secrets Operator (Vault/AWS SSM/GCP Secret Manager → K8s secrets), secret rotation strategies (database dynamic credentials/TLS cert-manager), detecting secrets in code (gitleaks/truffleHog/Semgrep), Airflow…
Skill Claude CodeCodex
Streaming pipeline reliability — exactly-once semantics (idempotent producer/transactional API/readcommitted/Kafka Streams EOS), at-least-once patterns with idempotent consumers, dead letter queue (DLQ) design with Kafka Connect SMT/custom error handler, poison message handling, reprocessing strategy, watermarks and…
Skill Claude CodeCodex
Terraform cost estimation — Infracost breakdown and diff commands (monthly cost per resource), PR cost comments (before/after change delta), budget thresholds (block PRs over cost limit), usage files for variable quantities (S3 GB/requests), AWS pricing data sources, cost tagging strategy, OpenTofu/Terraform cost…
Skill Claude CodeCodex
Terraform code review — module structure (main/variables/outputs/modules layout), variable validation blocks, type constraints, sensitive variables, locals vs variables, resource naming conventions, provider version pinning, data sources vs resources, module composition patterns, state management (remote backend…
Skill Claude CodeCodex
Terraform security scanning — tfsec static analysis (AWS/GCP/Azure misconfigurations), Checkov IaC policy checks (750+ rules, CIS benchmarks), tflint security rules, KICS scanner, S3 encryption/public access/versioning checks, IAM least-privilege patterns, security group rule review (0.0.0.0/0), KMS key rotation, VPC…
Skill Claude CodeCodex
Agentic control plane for data platform — MCP (Model Context Protocol) server exposing platform tools to AI agents (FastMCP/Python SDK), natural language platform operations (trigger DAG/create topic/run dbt/query Trino via LLM), Claude-based platform assistant with tool use, multi-agent platform governance (audit…
At most 3 mods per repository are shown here, and a mod shipped inside a plugin is left to that plugin's page — the rest are on their repository pages: