ableton-for-ai AGENTS.md

AI Agents Guide for Ableton-for-AI

This document provides essential technical context for AI agents working on this project.

Project Purpose

The goal is to provide a bridge between Ableton Live and AI-driven tools (like Mix-Engineers or LLMs) to enable automated mixing, analysis, and data extraction of music projects.

Core Components

1. osc_daemon.py (The Bridge)

• Role: A persistent background service that converts JSON-RPC over TCP into OSC over UDP.
• API Documentation: Whenever an agent needs to access Ableton via the osc_daemon, they MUST first consult the AbletonOSC README to ensure they are using the correct OSC addresses and parameter specifications.
• Ports:
  • Listens for JSON-RPC on 127.0.0.1:65432.
  • Sends OSC to Ableton on 11000.
  • Receives OSC from Ableton on 11001.
• Key Features:
  • Request/Response Matching: Uses a request_id and tracks pending futures to handle the asynchronous nature of OSC.
  • Wildcard Support: Can handle OSC wildcard queries (e.g., /live/track/get/*) and correctly dispatch multiple responses.
  • Tolerant Matching: Handles slight data type mismatches (int vs float) that AbletonOSC sometimes returns.
  • OSC Bundling: Supports grouped OSC messages in a single UDP packet for atomic execution and high performance.

2. ableton_client.py (The Interface)

• Role: High-level Python API for interacting with the osc_daemon.
• Key Operations:
  • get_tracks: Uses the highly efficient /live/song/get/track_data protocol for mass metadata retrieval.
  • extract_ableton_project_data: Quick extraction of project metadata, tracks, and device parameters.
  • get_available_stems / get_available_spectrograms: List available analysis results from previous runs.
  • analyze_stems: Independent audio analysis pipeline (spectrograms + energy) without project data.
• analyze_stems_and_extract_ableton_project_data: The full pipeline that gathers project data AND triggers deep audio analysis. Required to enable audio-related resources.
  • set_device_parameter / set_track_volume: Remote control methods to modify the Live session.
• Rate Limiting: Uses an asyncio.Semaphore (limit: 50) to prevent overwhelming the Ableton Remote Script while allowing high parallelism for parameter retrieval.