astronomer-airflow

Airflow 3.x on Astronomer

Operational reference for developing, running, and debugging Airflow 3.x DAGs in an Astronomer-managed local environment.

Project-specific config: If the project has a references/repo-config.md alongside this skill, read it first for Makefile targets, environment setup, file layout conventions, and other repo-specific details.

---

Environment Lifecycle (Astro CLI)

Action	Command
Start	astro dev start
Stop	astro dev stop
Restart	astro dev restart
Kill (full teardown)	astro dev kill
Run Airflow CLI commands	astro dev run <subcommand>
Run pytest inside container	astro dev run pytest tests/ -v

Default UI: http://localhost:8080 (credentials: admin / admin)

Port Conflicts

If port 8080 is already in use, stop the conflicting service first. Astro doesn't support custom webserver ports cleanly.

If port 5432 conflicts with a local Postgres, configure an alternate port in .astro/config.yaml.

---

DAG Development

DAG Decorator Pattern (Airflow 3.x)

from pathlib import Path
from airflow.sdk import dag
from pendulum import datetime

@dag(
    dag_id=Path(__file__).stem,    # Use filename as DAG ID
    start_date=datetime(2025, 1, 1, tz="UTC"),
    schedule=None,
    catchup=False,
    doc_md=__doc__,                # Module docstring as docs
    tags=["example"],
)
def my_pipeline() -> None:
    ...

my_pipeline()  # Must be called at module bottom

Key points:

• Import dag from airflow.sdk, not airflow.decorators
• Always call the decorated function at module level so Airflow discovers it
• Use Path(__file__).stem for dag_id to keep IDs in sync with filenames

.airflowignore

Airflow respects .airflowignore files in the dags/ directory to exclude paths from DAG parsing. If a DAG doesn't appear in the UI or dags list, check this file first.

DAG Loading and Reserializing

After creating or modifying a DAG:

1. Wait ~15 seconds for the scheduler to pick it up
2. Or force a reparse: astro dev run dags reserialize
3. Verify it loaded: astro dev run dags list | grep <dag_name>
4. Check for errors: astro dev run dags list-import-errors