MCP-Airflow-API copilot-instructions.md

Copilot Instructions for MCP-Airflow-API

Project Architecture

Core Pattern: MCP (Model Context Protocol) server exposing Apache Airflow REST API as LLM-friendly tools.

Key Components:

• src/mcp_airflow_api/airflow_api.py: 40+ @mcp.tool() decorators defining Airflow operations
• src/mcp_airflow_api/functions.py: Global session management with connection pooling
• src/mcp_airflow_api/prompt_template.md: Canonical English instructions for LLMs
• pyproject.toml: Entry point mcp-airflow-api = "mcp_airflow_mcp_main:main"

Transport Logic: Environment variable FASTMCP_TYPE controls stdio vs http mode:

transport_type = args.transport_type or os.getenv("FASTMCP_TYPE", "stdio")
if transport_type == "streamable-http":
    mcp.run(transport="streamable-http", host=host, port=port)  # Docker mode
else:
    mcp.run(transport='stdio')  # Local mode

Critical Patterns

Connection Pooling: Use global _airflow_session for performance:

# functions.py - persistent session with retry strategy
_airflow_session = requests.Session()  # Reused across all API calls

Pagination Strategy: All tools return {data, total_entries, limit, offset, has_more_pages, next_offset}:

# Standard pattern in airflow_api.py
def list_dags(limit=20, offset=0): 
    # Returns pagination metadata for large environments (1000+ DAGs)

Configuration Files: Multiple configs for different deployment modes:

• mcp-config.json.stdio: Local python -m mcp_airflow_mcp_main
• mcp-config.json.http: Docker "url": "http://host.docker.internal:18002/mcp"

Docker Multi-Service Pattern

docker-compose.yml orchestrates 3 services:

• mcp-server: FastMCP server (port 18002→8080 internal)
• mcpo-proxy: MCP-to-REST converter (Swagger at :8002/docs)
• open-webui: Web interface (port 3002)

Build Commands: Use versioned Docker images:

./build-mcp-server-docker-image.sh  # Builds call518/mcp-server-airflow-api:1.0.0