fabric-spark

fabric-spark is a skill for Claude Code from wardawgmalvicious/agent-config. It costs 223 tokens per session (2,800 once invoked), scanned A, original, MIT.

Instructions for using PySpark, a Python interface for Apache Spark, in Microsoft Fabric notebooks. They cover Fabric’s file, storage-path, identity, session-settings, and Lakehouse-table requirements.

In plain words
What is it for?
Use them when loading data through Fabric storage, addressing OneLake files, reading notebook identity details, tuning Spark sessions, managing files, or working with Delta Lake tables.
Why use it?
They prevent common Fabric errors, such as trying to fetch external web data directly or confusing workspace identity settings with Spark tuning settings.

Skill for Claude Code

Written for Claude Code: disable-model-invocation in frontmatter.

Good fit Use them when loading data through Fabric storage, addressing OneLake files, reading notebook identity details, tuning Spark sessions, managing files, or working with Delta Lake tables.

Compare 6 skills from other repositories ↓
Install with agentmods
npx agentmods add skills/wardawgmalvicious/agent-config/fabric-spark
Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

Any agent
npx skills add wardawgmalvicious/agent-config --skill fabric-spark
Clone the repo
git clone --depth 1 https://github.com/wardawgmalvicious/agent-config

Made for: Claude Code.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for fabric-spark

README.md
[![agentmods](https://agentmods.dev/badge/skills/wardawgmalvicious/agent-config/fabric-spark/github.svg)](https://agentmods.dev/skills/wardawgmalvicious/agent-config/fabric-spark)
Your own site
<a href="https://agentmods.dev/skills/wardawgmalvicious/agent-config/fabric-spark"><img src="https://agentmods.dev/badge/skills/wardawgmalvicious/agent-config/fabric-spark/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

agentmods 80×15 button for fabric-spark

Your own site · 80×15
<a href="https://agentmods.dev/skills/wardawgmalvicious/agent-config/fabric-spark"><img src="https://agentmods.dev/badge/skills/wardawgmalvicious/agent-config/fabric-spark.svg" alt="Reviewed on agentmods" width="80" height="20"></a>
Per session 223 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 2,800 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 1 finding. A grade says what 26 rules found in the file — not that it is safe.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00223 $0.02800
Opus 5 $0.00112 $0.01400
Sonnet 5 $0.00045 $0.00560
Haiku 4.5 $0.00022 $0.00280

Measured 2d ago against content hash cfe9364b4c66, method: parsed. Prices are Anthropic first-party input rates as of 2026-09-12, from the pricing page.

Security

Grade A, and why

fabric-spark scanned grade A with 1 finding against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Makes network callslowCapability

Not a fault in itself. Listed so you know the mod talks to something, and to what.

- Fabric Spark cannot access arbitrary external HTTP/HTTPS URLs — land data in lakehouse `Files/` first (via pipeline Copy activity, OneLake API, or curl)
skills/fabric/fabric-spark/SKILL.md · 169 lines

How it starts

The opening of the file, as written. The whole thing — 169 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Spark / PySpark in Fabric

Key Constraints

  • Fabric Spark cannot access arbitrary external HTTP/HTTPS URLs — land data in lakehouse Files/ first (via pipeline Copy activity, OneLake API, or curl)
  • Use abfss:// URI format for OneLake paths in Spark: abfss://{workspace}@onelake.dfs.fabric.microsoft.com/{item}.Lakehouse/{path}
  • Use workspace GUIDs (not names) in ABFS URIs — spaces are not allowed
  • mssparkutils for Fabric-specific notebook operations (credentials, secrets, file management)
  • Use Delta Lake format for all Lakehouse tables

Runtime Context vs Spark Session Config

Two different things that are often confused:

Need API
Workspace / item identity (workspace ID + name, notebook ID + name, default lakehouse ID + name, userId) notebookutils.runtime.context["currentWorkspaceId"] (etc.) — a dict, documented public API, works in pure-Python notebooks
Spark session tuning (shuffle partitions, AQE, Delta settings, case sensitivity) spark.conf.set(...) / spark.conf.get(...)

spark.conf.get("trident.workspace.id") also returns the workspace ID but is internal Spark conf, not documented surface, and is unavailable in pure-Python notebooks. Prefer notebookutils.runtime.context for identity lookups; reserve spark.conf.* for session tuning.

Lakehouse Setup

  • enableSchemas is set at lakehouse creation time only — cannot be retrofitted. Without it the lakehouse only has the default dbo schema and you must recreate to gain named schemas. Set via creationPayload: { "enableSchemas": true } on POST /workspaces/{ws}/items (see fabric-rest-api skill).
  • Schemas use lowercase names by convention (bronze/silver/gold for medallion). DROP SCHEMA <name> CASCADE removes the schema with all its tables.
  • Cross-lakehouse Spark SQL uses 3-part names: lakehouse.schema.table for same-workspace, workspace.lakehouse.schema.table for cross-workspace. Verify access permissions on each lakehouse.
  • Lakehouse delete cascades irreversibly: SQL Endpoint deleted, all OneLake data permanently removed, shortcuts pointing in become inaccessible, dependent notebooks fail at runtime.
  • Shortcuts as definition payload: when authoring a Lakehouse via REST, shortcuts.metadata.json is an array of {name, path, target} objects. Supported target.type values: OneLake, AdlsGen2, AmazonS3, GoogleCloudStorage, S3Compatible, Dataverse. Each target type has its own connection properties (see fabric-rest-api skill).

Read the full file on GitHub · 169 lines

Files

What ships with it

1 file beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 2d ago Changed cfe9364b4c66
  2. 8d ago Changed · +3 lines 16b18666f309
  3. 12d ago First seen · 166 lines · 223 tokens per session scan A d277b959942c

Subscribe to this mod's changes

fabric-spark is a skill published in the GitHub repository wardawgmalvicious/agent-config (1 stars, last pushed yesterday), licensed MIT. It adds 223 tokens to every session and 2,800 once invoked, about $0.0011 per session on Opus 5. A static security scan graded it A with 1 finding (makes network calls). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.

Related

Other skills, from other repositories

azure-mgmt-fabric-py

Azure Fabric Management SDK for Python. Use for managing Microsoft Fabric capacities and resources. Triggers: "azure-mgmt-fabric", "FabricMgmtClient", "Fabric capacity", "Microsoft Fabric", "Power BI capacity".

microsoft/skills · 51 tokens

matlab-integrate-pytorch-vision

Creates MATLAB interfaces to Python image processing and computer vision models from GitHub repositories or pip-installable packages using MPyReq. Use when asked to interface MATLAB with a Python CV/image model (segmentation, depth estimation, object detection, image generation, super-resolution, etc.), given a GitHub…

matlab/matlab-agentic-toolkit · 132 tokens

fabric-pandas-perf-remediate

Troubleshoot and optimize pandas performance in Microsoft Fabric Spark notebooks. Use when diagnosing slow pandas operations, toPandas() out-of-memory errors, pandas API on Spark (pyspark.pandas) bottlenecks, DataFrame conversion failures, collect() memory issues, driver memory exhaustion, notebook cell timeouts, or…

PatrickGallucci/fabric-skills · 107 tokens

scikit-learn

Machine learning in Python with scikit-learn. Use when working with supervised learning (classification, regression), unsupervised learning (clustering, dimensionality reduction), model evaluation, hyperparameter tuning, preprocessing, or building ML pipelines. Provides comprehensive reference documentation for…

MarieLynneBlock/arcanum-artifex · 68 tokens

scikit-learn-pipelines-and-column-transformers

Skill "scikit-learn-pipelines-and-column-transformers" from MarieLynneBlock/arcanum-artifex, covering what this skill does, when to use it, instructions, output format and examples.

MarieLynneBlock/arcanum-artifex · 35 tokens

accelerate

Run PyTorch training across GPUs with minimal changes.

NousResearch/hermes-agent · 13 tokens