data-data-lakehouse

Data Data Lakehouse

Purpose

Design lakehouse architectures that merge data lake flexibility with warehouse reliability. Implement medallion architecture for data quality progression, Unity Catalog for governance, Delta Sharing for data collaboration, and multi-cloud deployment patterns.

Agent Protocol

Trigger

Exact user phrases: "lakehouse", "medallion architecture", "bronze", "silver", "gold", "Databricks", "Unity Catalog", "Delta Sharing", "Apache Paimon", "multi-cloud lakehouse", "open formats", "lakehouse governance", "data mesh lakehouse".

Input Context

Before activating, verify:

• Cloud provider (AWS, Azure, GCP, multi-cloud)
• Lakehouse platform (Databricks, AWS EMR, Azure Synapse, GCP Dataproc)
• Table format (Delta, Iceberg, Paimon)
• Data sources volume and types
• Number of data producers and consumers
• Security requirements (RBAC, column mask, row filter)
• Data sharing requirements (internal teams, external partners)

Output Artifact

Lakehouse architecture with medallion layers, Unity Catalog configuration, and platform deployment specs.

Response Format

Lakehouse Platform: {Databricks | EMR + Iceberg | Synapse + Delta | Dataproc + Iceberg}
Catalog: {Unity Catalog | Hive Metastore | AWS Glue | Nessie}
Table Format: {Delta | Iceberg | Paimon}
Medallion Layers: Bronze (raw), Silver (cleaned), Gold (aggregated)
Sharing: {Delta Sharing | open | proprietary}

# Unity Catalog metastore config
# Medallion pipeline YAML

No preamble. No postamble. No explanations. No filler/hedging/transitions. Compress output — why use many token when few do trick.

Completion Criteria

• Medallion layers defined with data flow and transformations
• Unity Catalog or equivalent metastore configured
• RBAC and column-level security defined
• Delta Sharing setup for cross-team/partner data access
• Table format selected with interoperability plan
• Multi-cloud or cross-region replication strategy
• Data quality checks at each medallion layer
• Platform deployment topology (compute, storage, catalog)