rehashing-datavault4dbt-entities

Rehashing datavault4dbt entities

When you change anything that affects hash output — the algorithm (MD5→SHA2), hash_datatype, hashdiff_use_trim, or case-sensitivity vars — every hashkey and hashdiff in the Raw Data Vault must change too, or referential integrity breaks. You have two options:

1. Full refresh the RDV (only if the full source history is still available to reload), or
2. Rehash existing entities in place with the package's rehashing macros (when history is not reloadable).

This skill is about option 2.

v1 → v2.0.0 upgrade. v2.0.0 unifies/corrects hash standardization: with default settings, hashkeys are now UPPER-normalized and hashdiffs are no longer UPPER-normalized, plus adapter-specific changes (Oracle/Redshift concat-string alignment, BigQuery MD5 fix, Fabric/Exasol standardization). Hash values differ from v1, so crossing this boundary on an existing vault requires a full refresh or a rehash.

The safe workflow (always)

1. Start small — rehash one test entity first and verify the configuration and logic.
2. Overwrite, don't drop — set overwrite_hash_values: true and keep drop_old_values: false. The macros rename the old columns with a _deprecated suffix instead of deleting them.
3. Validate — compare the new hash columns against the _deprecated columns.
4. Clean up — only after validation, drop the deprecated columns. The dbt log prints the dict of columns to drop; use it to build a cleanup model or call datavault4dbt.custom_alter_relation_add_remove_columns.

Single-entity rehash (surgical)

Run a one-off operation against a single hub/link/satellite:

dbt run-operation rehash_single_hub --args '{
    hub: customer_h,
    hashkey: HK_CUSTOMER_H,
    business_keys: C_CUSTKEY,
    overwrite_hash_values: true
}'

Bulk rehash by entity type (YAML-driven)

Create a dedicated model that calls the bulk macro with YAML metadata:

-- models/rehash/rehash_hubs.sql
{{ config(materialized='view') }}
{% set hub_yaml %}
config:
    overwrite_hash_values: true
hubs:
    - name: customer_h
      hashkey: hk_customer_h
      business_keys: [c_custkey]
    - name: order_h
      hashkey: hk_order_h
      business_keys: [order_id]
{% endset %}

{{ datavault4dbt.rehash_hubs(hub_yaml=hub_yaml, drop_old_values=false) }}

SELECT 'success' as status