wren-dlt-connector

wren-dlt-connector

Connect SaaS data to Wren Engine for SQL analysis — from zero to a verified, queryable project in one conversation.

Who this is for

Data analysts who know SQL and some Python, but may not have used dlt or Wren before. Explain concepts briefly when they first appear, but don't over-explain things a SQL-literate person would already know.

Overview

This skill walks through a four-phase workflow:

1. Extract — Use dlt (data load tool) to pull data from a SaaS API into a local DuckDB file
2. Model — Introspect the DuckDB schema and auto-generate a Wren semantic project (YAML models, relationships, profile)
3. Build & Verify — Build the project and run actual SQL queries to confirm everything works end-to-end
4. Handoff — Show the user their data and next steps

The user might enter at any phase. Ask which phase they're starting from — they may already have a .duckdb file and just need phases 2–4.

The goal is a project that actually queries successfully, not just files that look correct. Always run the verification step before declaring success.

Critical: DuckDB catalog naming

When wren engine connects to a DuckDB file, it ATTACHes it using the filename (without .duckdb extension) as the catalog alias:

ATTACH DATABASE 'stripe_data.duckdb' AS "stripe_data" (READ_ONLY)

This means every model's table_reference.catalog must equal the DuckDB filename stem. If the file is hubspot.duckdb, the catalog is hubspot. If it's my_pipeline.duckdb, the catalog is my_pipeline.

Getting this wrong causes "table not found" errors at query time. The introspect_dlt.py script handles this automatically.

Critical: Type normalization

Column types must be normalized using wren SDK's type_mapping.parse_type() function, which uses sqlglot to convert database-specific types (like DuckDB's HUGEINT, TIMESTAMP WITH TIME ZONE) into canonical SQL types that wren-core understands. Do not hardcode type mappings — always delegate to parse_type(raw_type, "duckdb").