apache spark skills

mlflow/mlflow

Skill Claude CodeCodex

Analyze failed GitHub Action jobs. Takes one or more GitHub URLs (job, workflow-run, or PR) and summarizes each failure with root cause and log paths.

mlflow/mlflow

Skill Claude CodeCodex

Review a pull request and emit a validated review payload.

mlflow/mlflow

Skill Claude CodeCodex

Upload one or more local images or videos to GitHub and get back a user-attachments URL for each, to embed in a PR body, issue, or comment. Use when asked to attach screenshots or screen recordings.

EmbrasureAI/spark-observability-skills

Skill Claude CodeCodex

Diagnose slow, expensive, or regressed Apache Spark and PySpark applications by comparing runtime evidence against a healthy run. Use for long stages, stragglers, skew, shuffle, spill, garbage collection, poor parallelism, small files, slow scans, scheduler delay, executor imbalance, and unexplained compute-cost…

EmbrasureAI/spark-observability-skills

Skill Claude CodeCodex

Diagnose failed Apache Spark and PySpark applications from History Server evidence, logs, and cluster-manager state. Use for driver or executor crashes, out-of-memory errors, fetch failures, task exceptions, timeouts, repeated retries, aborted stages, and intermittent production failures.

EmbrasureAI/spark-observability-skills

Skill Claude CodeCodex

Optimize Apache Spark SQL and DataFrame queries using the final Adaptive Query Execution plan and runtime statistics rather than source code alone. Use to reduce runtime, shuffle, spill, scan cost, skew, join amplification, Python UDF overhead, poor partitioning, or unnecessary work while preserving query semantics.

vaquarkhan/data-engineering-agent-skills

Skill Claude CodeCodex

Guides agents through schema-registry-backed event contracts. Use when managing Avro, Protobuf, or JSON Schema for event streams, compatibility policies, producer and consumer evolution, or contract enforcement in messaging systems.

vaquarkhan/data-engineering-agent-skills

Skill Claude CodeCodex

Guides agents through data-contract testing using schema registries and compatibility checks. Use when validating event contracts, stream schema evolution, consumer compatibility, or release gates for schema-managed systems.

vaquarkhan/data-engineering-agent-skills

Skill Claude CodeCodex

Guides agents through Debezium and Kafka Connect CDC workflows. Use when streaming database changes into Kafka topics, managing connectors, snapshots, schema evolution, or downstream CDC consumers.

Galius5136/databricks-spark-3.5-cert-prep

Skill Claude CodeCodex

Knowledge base from "Learning Spark, 2nd Edition" by Damji, Wenig, Das & Lee. Use when working with Apache Spark / PySpark, preparing for the Databricks Certified Associate Developer for Apache Spark exam, or applying the authors' frameworks for Spark architecture, DataFrame API, Spark SQL, Structured Streaming…

Galius5136/databricks-spark-3.5-cert-prep

Skill Claude CodeCodex

Knowledge base for Pandas API on Spark (pyspark.pandas) — Apache Spark 3.5. Use when preparing for Sec 7 objective 'Explain advantages of using Pandas API on Spark', migrating pandas code to scale on Spark, or configuring pyspark.pandas options (default index type, opsondiffframes, checkpointing). Sources…

Galius5136/databricks-spark-3.5-cert-prep

Skill Claude CodeCodex

Knowledge base for Spark 3.1→3.5 updates not covered by the Learning Spark 2nd Ed book. Use for: (A) Pandas UDF and Arrow APIs (exam Sec 7 obj 2 'Create and invoke Pandas UDF'); (B) new built-in functions added in 3.3-3.5 (exam Sec 3); (C) AQE evolution post-3.0 — default-on, new sub-configs, SMJ→SHJ conversion (exam…