Speech-To-Text-With-Databricks: Skill for Claude Code

Synthetic Data Generation

Generate realistic, story-driven synthetic data for Databricks using Python with Faker and Spark.

Common Libraries

These libraries are useful for generating realistic synthetic data:

• faker: Generates realistic names, addresses, emails, companies, dates, etc.
• holidays: Provides country-specific holiday calendars for realistic date patterns

These are typically NOT pre-installed on Databricks. Install them using execute_databricks_command tool:

• code: "%pip install faker holidays"

Save the returned cluster_id and context_id for subsequent calls.

Workflow

1. Write Python code to a local file in the project (e.g., scripts/generate_data.py)
2. Execute on Databricks using the run_python_file_on_databricks MCP tool
3. If execution fails: Edit the local file to fix the error, then re-execute
4. Reuse the context for follow-up executions by passing the returned cluster_id and context_id

Always work with local files first, then execute. This makes debugging easier - you can see and edit the code.

Context Reuse Pattern

The first execution auto-selects a running cluster and creates an execution context. Reuse this context for follow-up calls - it's much faster (~1s vs ~15s) and shares variables/imports:

First execution - use run_python_file_on_databricks tool:

• file_path: "scripts/generate_data.py"

Returns: { success, output, error, cluster_id, context_id, ... }

Save cluster_id and context_id for follow-up calls.

If execution fails:

1. Read the error from the result
2. Edit the local Python file to fix the issue
3. Re-execute with same context using run_python_file_on_databricks tool:
   • file_path: "scripts/generate_data.py"
   • cluster_id: "<saved_cluster_id>"
   • context_id: "<saved_context_id>"

Follow-up executions reuse the context (faster, shares state):

• file_path: "scripts/validate_data.py"
• cluster_id: "<saved_cluster_id>"
• context_id: "<saved_context_id>"