Skill Claude CodeCodex
Terraform security scanning — tfsec static analysis (AWS/GCP/Azure misconfigurations), Checkov IaC policy checks (750+ rules, CIS benchmarks), tflint security rules, KICS scanner, S3 encryption/public access/versioning checks, IAM least-privilege patterns, security group rule review (0.0.0.0/0), KMS key rotation, VPC…
Skill Claude CodeCodex
Agentic control plane for data platform — MCP (Model Context Protocol) server exposing platform tools to AI agents (FastMCP/Python SDK), natural language platform operations (trigger DAG/create topic/run dbt/query Trino via LLM), Claude-based platform assistant with tool use, multi-agent platform governance (audit…
Skill Claude CodeCodex
Data platform self-service API — FastAPI REST API for platform operations (Kafka topic CRUD/Airflow DAG trigger/dbt job run/Trino query execution), OAuth2+JWT authentication with team-based RBAC, async job tracking with status polling, rate limiting per team, OpenAPI spec generation, Terraform provider for API…
Skill Claude CodeCodex
Internal Developer Platform (IDP) for data engineering — Backstage Software Catalog (catalog-info.yaml for pipelines/datasets/services), Software Templates (scaffold new DAG/dbt project/Kafka topic), TechDocs integration, golden path enforcement, Port.io alternative, self-service data pipeline provisioning, platform…
Skill Claude CodeCodex
Airflow + StarRocks backfill patterns — historical partition reprocessing, date range loops with INSERT OVERWRITE, Broker Load replay for S3 partitions, safe backfill with maxactiveruns=1, partial backfill resume from checkpoint, backfill idempotency via label strategy, clearing downstream tasks before rerun, parallel…
Skill Claude CodeCodex
Airflow CDC orchestration for StarRocks — watermark-based incremental sync, Routine Load health check DAG, Flink job submission via REST API, offset lag alerting, dead letter queue reprocessing, multi-table incremental sync with dependency graph, schema change detection and pipeline pause/resume.
Skill Claude CodeCodex
Airflow + StarRocks data quality gates — post-load row count validation, freshness checks (MAX(updatedat) vs expected), null explosion detection, duplicate key audit on Primary Key tables, volume anomaly detection (z-score vs 7-day avg), SODA-style SQL quality checks embedded in DAG, quarantine pattern for bad…
Skill Claude CodeCodex
Airflow + StarRocks ETL best practices — idempotent DAG design (label strategy/partition overwrite), retry with backoff, SLA callbacks, dynamic partition creation, duplicate prevention with MERGE/INSERT OVERWRITE, DAG-level concurrency controls, dependency ordering, catchup safety, data lineage tagging.
Skill Claude CodeCodex
Airflow + StarRocks pipeline orchestration — MySqlHook for DDL/DML, Broker Load trigger + poll pattern, Stream Load via HTTP operator, Routine Load job lifecycle (pause/resume/stop), sensor for load completion, ANALYZE TABLE post-load, partition-aware DAG templating, Airflow variables for StarRocks connection config…
Skill Claude CodeCodex
Skill "dbt-starrocks-models" from ivanshamaev/de-agent-skills, covering dbt + starrocks models, when to use, installation and setup, dbt-starrocks requires dbt-core >= 1.5.0 and profiles.yml.
Skill Claude CodeCodex
The most impactful optimization: ensure incremental models only scan new partitions.
Skill Claude CodeCodex
-- Create dbt user CREATE USER dbtprod IDENTIFIED BY 'strongpasswordhere'.