Open-source agent skills for debugging and optimizing Apache Spark workloads
EmbrasureAI/spark-observability-skills
Skill Claude CodeCodex
Diagnose slow, expensive, or regressed Apache Spark and PySpark applications by comparing runtime evidence against a healthy run. Use for long stages, stragglers, skew, shuffle, spill, garbage collection, poor parallelism, small files, slow scans, scheduler delay, executor imbalance, and unexplained compute-cost…
EmbrasureAI/spark-observability-skills
Skill Claude CodeCodex
Diagnose failed Apache Spark and PySpark applications from History Server evidence, logs, and cluster-manager state. Use for driver or executor crashes, out-of-memory errors, fetch failures, task exceptions, timeouts, repeated retries, aborted stages, and intermittent production failures.
EmbrasureAI/spark-observability-skills
Skill Claude CodeCodex
Optimize Apache Spark SQL and DataFrame queries using the final Adaptive Query Execution plan and runtime statistics rather than source code alone. Use to reduce runtime, shuffle, spill, scan cost, skew, join amplification, Python UDF overhead, poor partitioning, or unnecessary work while preserving query semantics.