spark-submit

spark-submit is a skill for Claude Code, Codex from OleanderHQ/claude-plugin. It costs 55 tokens per session (1,881 once invoked), scanned A, original, Apache-2.0.

A guide for submitting and managing Apache Spark jobs on Oleander, a Spark execution service. It covers uploading job files, starting runs, monitoring them, stopping them, and using MCP, the command line, or a TypeScript SDK.

In plain words
What is it for?
Use it to upload PySpark scripts, submit Spark or SQL jobs, monitor runs, inspect artifacts, abort jobs, and automate submissions through supported interfaces.
Why use it?
It provides the required workflow and parameter conventions for running Spark jobs without guessing how artifacts, SQL, or job states should be handled.

Skill for Claude CodeCodex

Part of the oleander plugin — 7 skills, 1 MCP server shipped together

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add skills/oleanderhq/claude-plugin/spark-submit
Any agent
npx skills add OleanderHQ/claude-plugin --skill spark-submit
Clone the repo
git clone --depth 1 https://github.com/OleanderHQ/claude-plugin

Made for: Claude Code, Codex.

Or install oleander, the plugin that ships this one along with the rest of its 7 skills, 1 MCP server.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for spark-submit

README.md
[![agentmods](https://agentmods.dev/badge/skills/oleanderhq/claude-plugin/spark-submit.svg)](https://agentmods.dev/skills/oleanderhq/claude-plugin/spark-submit)
Your own site
<a href="https://agentmods.dev/skills/oleanderhq/claude-plugin/spark-submit"><img src="https://agentmods.dev/badge/skills/oleanderhq/claude-plugin/spark-submit.svg" alt="Measured on agentmods" height="20"></a>
Per session 55 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 1,881 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00055 $0.01881
Opus 5 $0.00028 $0.00941
Sonnet 5 $0.00011 $0.00376
Haiku 4.5 $0.00006 $0.00188

Measured 4d ago against content hash 2e53eadd376f, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

spark-submit scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 4d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

skills/spark-submit/SKILL.md · 246 lines

How it starts

The opening of the file, as written. The whole thing — 246 lines — stays where its author put it; the contents beside it link to each section on GitHub.

oleander Spark Submit

Use this skill when submitting Spark jobs to oleander, monitoring run state, or building automated workflows that execute Spark jobs.

MCP submission

Use these tools when working through the oleander MCP server (Claude Code, Cursor, etc.). Tool names are underscore-delimited — for example spark_jobs_submit, not spark.jobs.submit or oleander_submit_spark_job.

For SQL that writes to a table, use query_submit, which routes the query and sizes compute automatically. Use spark_sql_submit only when the run needs a durable job name or explicit machine types. See the lake-query skill.

Artifacts

Before submitting a PySpark script, check whether it is already uploaded:

  1. spark_artifacts_list — list ready artifacts and versions
  2. If missing or the user asks for a new version: spark_artifacts_upload with entrypoint, entrypoint_content_base64, and confirm: true
  3. To inspect source before running: spark_artifacts_get

Pass the artifact basename from the list (for example my_job.py) as properties.entrypoint in spark_jobs_submit, not the storage URI.

Submit a job

Call spark_jobs_submit with:

  • namespace and name — job identity
  • properties.entrypoint — artifact basename
  • properties.entrypointArguments — optional args passed to the script
  • properties.driverMachineType, properties.executorMachineType, properties.executorNumbers — sizing (defaults: spark.1.b, 2 executors)
  • confirm: true

The response includes runId and initial state (typically SUBMITTED). Cluster is always oleander-managed serverless Spark.

Monitor a run

Poll until the run reaches a terminal state (COMPLETE, FAIL, or ABORT):

  1. jobs_runs_get with the run_id from submit
  2. Retry if the run is not visible yet — freshly submitted runs can take a moment to appear

Related tools after submit:

  • jobs_runs_list — recent runs for a namespace + job name
  • jobs_logs_get — driver and executor logs
  • jobs_traces_get — OpenTelemetry traces
  • jobs_cost_get — run cost breakdown
  • jobs_lineage_get — datasets read and written

Read the full file on GitHub · 246 lines

Files

What ships with it

1 file beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 4d ago First seen · 246 lines · 55 tokens per session scan A 2e53eadd376f

Subscribe to this mod's changes

spark-submit is a skill published in the GitHub repository OleanderHQ/claude-plugin (4 stars, last pushed 25d ago), licensed Apache-2.0. It adds 55 tokens to every session and 1,881 once invoked, about $0.0003 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.

Related

Other skills, from other repositories

cloud-architect

Designs cloud architectures, creates migration plans, generates cost optimization recommendations, and produces disaster recovery strategies across AWS, Azure, and GCP. Use when designing cloud architectures, planning migrations, or optimizing multi-cloud deployments. Invoke for Well-Architected Framework, cost…

Jeffallan/claude-skills · 71 tokens

rule

Skill "rule" from x-cmd/x-cmd, covering rule — rules to rule, 核心命令, 规则文件格式, 示例 and 验证规则文件格式.

x-cmd/x-cmd · 0 tokens

azmon-mirroredcatalogs-operations-cli

Brings Azure Monitor, Application Insights, and Log Analytics telemetry into Fabric as Eventhouse external delta tables and correlates it with business data. Use to onboard observability data, judge whether latency or availability affected revenue, or build a Real-Time dashboard and Operations Agent over it.

microsoft/skills-for-fabric · 66 tokens

atmos-auth

Authentication and identity management: providers (SSO/SAML/OIDC/GCP/Atmos Pro), identities, keyring, identity chaining, login/exec/shell/console, and github/sts for private GitHub access.

cloudposse/atmos · 48 tokens

terraform-search-import

Discover existing cloud resources using Terraform Search queries and bulk import them into Terraform management. Use when bringing unmanaged infrastructure under Terraform control, auditing cloud resources, or migrating to IaC.

hashicorp/agent-skills · 39 tokens

x-scorecard

OpenSSF Scorecard for assessing open source project security. Check security best practices and compliance. Dependency: This is an x-cmd module. Install x-cmd first (see x-cmd skill for installation options). see x-cmd skill for installation.

x-cmd/x-cmd · 57 tokens