legacy-to-dbt-migration-foundations

Legacy-to-dbt Migration Foundations

This is a shared reference library. It is not a standalone workflow. The three migration skills — migrating-informatica-to-dbt, migrating-talend-to-dbt, and migrating-stored-procedures-to-dbt — all run the same 8-step migration workflow and defer the common steps to the references below. When one of those skills points you here, open the specific reference named for that step.

Every migration, regardless of source, must:

1. Map the original workload — inventory every unit of work before writing any dbt code.
2. Translate it into dbt with best practices — ref()/source(), tests, docs, contracts.
3. Validate the result against real warehouse data — prove parity, not just that it compiles.
4. Right-size for the target cloud — ask which warehouse is in use and pick cost-aware materializations.
5. Report cost (legacy run vs dbt dev run) and coverage (target ≥ 95%).

Core principle: compilation is not correctness. dbt compile (free, no warehouse queries) is the fast iteration gate; data parity against the warehouse is the proof the migration preserved business logic. Never declare a migration done on a clean compile alone.

Suggest, don't decide — put every up-front choice to the migrator. A handful of decisions reshape the whole migration (file layout, SQL dialect, what validation can reach), so getting them wrong means redoing dozens of files. Before writing any models, present them as one consolidated set of choices, each with your recommended default + a one-line why and the alternatives, then wait for the migrator's answer:

1. Target platform (+ dev target) — Snowflake / Databricks / BigQuery / Redshift; Fusion vs Core.
2. Target modeling approach — Data Vault / Kimball / Star (or a faithful layered port).
3. Packages vs self-contained macros — external hub packages, or skill-written macros.
4. Landing spot — a new standalone dbt project, or fold into an existing one.

Recommend based on the signals (e.g. "the workload looks Kimball, so I suggest Kimball"), but never silently default even when a choice looks obvious — offer it, recommend, and let them choose. This is the Step 0 / Step 2 gate; treat it as blocking.

Teach as you migrate — assume the migrator may be new to dbt. The person may know their legacy tool well but not dbt. Whenever you introduce a dbt concept — a model, ref()/source(), a materialization (view / table / incremental / ephemeral), a snapshot (SCD history), tests, contracts, Fusion — explain it in one plain sentence and point to dbt-concepts-explained.md. Explain the reasons, not just the mechanics: why this table is incremental, why history uses a snapshot, why this target modeling approach (see target-modeling.md). The migration should leave the person understanding their dbt project, not just possessing it.

Packages: ask first, provision on demand, hub-only. The skill doesn't bundle or assume packages. At Step 0 it asks the migrator whether to use external dbt packages or stay self-contained (skill-written macros). If packages are allowed, it installs only from hub.getdbt.com (never git/tarball/private sources) and only what each step detects it needs — dbt_utils, codegen, audit_helper, dbt_expectations, dbt_project_evaluator, datavault4dbt/dbt_date per modeling approach — pinned to a Fusion-badged version, then dbt deps. It never hand-rolls boilerplate a hub package solves (when packages are allowed), and never bloats packages.yml. If the migrator declines packages, it generates the equivalent macros instead. See dbt-packages.md.