airflow-cross-dag-to-orchestra

Airflow Cross-DAG Triggering → Orchestra

Overview

Airflow cross-DAG patterns — TriggerDagRunOperator, ExternalTaskSensor, SubDagOperator, Datasets — all have Orchestra equivalents. Orchestra provides two mechanisms:

1. TRIGGER_PIPELINE task — an explicit pipeline step that triggers another pipeline and waits for it to complete. Equivalent to TriggerDagRunOperator(wait_for_completion=True).

2. trigger_events: block — event-driven triggering at the pipeline root. When an upstream pipeline completes, this pipeline starts automatically. Equivalent to ExternalTaskSensor + DAG scheduling, but cleaner.

---

TriggerDagRunOperator → ORCHESTRA + TRIGGER_PIPELINE

# Airflow
trigger_downstream = TriggerDagRunOperator(
    task_id="trigger_reporting",
    trigger_dag_id="daily_reporting",
    conf={"date": "{{ ds }}", "env": "prod"},
    wait_for_completion=True,
    execution_date="{{ execution_date }}",
)

# Orchestra
task-001:
  integration: ORCHESTRA
  integration_job: TRIGGER_PIPELINE
  name: trigger_reporting
  parameters:
    pipeline_id: "uuid-of-daily-reporting-pipeline"   # UUID from Orchestra UI
    run_inputs:                                         # optional — equivalent to conf={}
      date: "${{ ORCHESTRA.CURRENT_TIME }}"
      env: prod
    branch: null                                        # optional — git branch override
  depends_on: []
  condition: null
  tags: []

TRIGGER_PIPELINE parameters:

Parameter	Required	Notes
pipeline_id	✅	UUID of the pipeline to trigger (find in Orchestra URL)
run_inputs	❌	Key-value dict passed as inputs to the triggered pipeline
branch	❌	Git branch to use for the triggered pipeline run

Key difference from Airflow: TRIGGER_PIPELINE always waits for the triggered pipeline to complete before proceeding. There is no wait_for_completion=False equivalent.

---

ExternalTaskSensor → trigger_events: (preferred)