aidp-build-dag

aidp-build-dag — build the migration manifest

The execution DAG is what the migrator reads to know which notebooks to migrate, in what order, with what dependencies. Build it once per workload.

When to use

• User asks "what would migrate", "show me the dependency tree", "build the manifest".
• Before any aidp-migrate-job invocation against a new workload.
• After changing the Databricks workspace path or workflow ID.

Two entry points

The migrator ships two DAG builders. Pick based on input shape:

Input	Entrypoint
Path-based: a folder of .ipynb / .py notebooks the user wants migrated	${CLAUDE_PLUGIN_ROOT}/engine/scripts/build_dag.py
Workflow-based: a Databricks Job ID whose tasks the user wants migrated, preserving the task DAG	${CLAUDE_PLUGIN_ROOT}/engine/scripts/build_dag_from_workflow.py

Path-based invocation

python3 ${CLAUDE_PLUGIN_ROOT}/engine/scripts/build_dag.py \
  --root "<databricks-workspace-path>" \
  --job-name "<MyJob>" \
  --output reports/<MyJob>_manifest.json

• --root — Databricks workspace folder containing the entry notebooks. The script walks every *.ipynb / *.py under this prefix.
• --job-name — a name for the manifest (used as the output-base subdirectory).
• --output — manifest write path.

The builder follows %run chains AND dbutils.notebook.run(...) calls to build a topo-ordered DAG. It also flags transitive deps so Pass-1 knows which notebooks to migrate code-only first.

Workflow-based invocation

python3 ${CLAUDE_PLUGIN_ROOT}/engine/scripts/build_dag_from_workflow.py \
  --job-id <databricks-job-id> \
  --output reports/<MyJob>_manifest.json

This pulls the Job's task definitions via the Databricks Jobs REST API and converts depends_on task edges into the manifest's DAG. Use when the user wants the AIDP migration to mirror the Databricks Workflow shape (vs. just inferring dependencies from %run).