bauplan-data-quality-checks

Data Quality Checks

This skill writes data quality check code. It produces one of two things:

1. Pipeline expectations — an expectations.py file using @bauplan.expectation() that runs as part of bauplan run.
2. Ingestion validation — a validate_import() function using the bauplan SDK, embedded in a WAP script between import_data() and merge_branch().

Output is always working code. Not reports, not profiling summaries, not markdown.

This skill is invoked by the bauplan-data-pipeline and bauplan-safe-ingestion workflow skills when they determine that quality checks are needed. It can also be invoked directly by the user. Either way, the skill needs to know three things before it can write code:

1. What table(s) and branch — namespace.table_name and the ref to validate against
2. What context — pipeline or ingestion, which determines the code form
3. What to check — this comes in one of two forms:
   • User specifications: the user states checks directly ("user_id must be unique, age must be positive"). Translate to code.
   • Pipeline code: a models.py exists that consumes the table. Read it, derive checks, propose them to the user for confirmation, then write code.

If the skill is invoked without enough information, ask for what's missing. But ask for specifics — "which columns and what properties?" — not for a general description of the pipeline's purpose.

CRITICAL: Branch Safety

NEVER run checks or pipelines on main. All validation targets a development or import branch.

Branch naming convention: <username>.<branch_name>. Get your username with bauplan info.

Environment Setup

Before writing any Python, check whether the project uses uv (look for pyproject.toml or uv.lock). If so, use uv run python to execute scripts and uv add to install packages. Otherwise, use the system python and pip install.

Ensure the required packages are installed:

• bauplan (the Bauplan Python SDK — required)
• polars (if custom expectations need DataFrame operations — zero-copy Arrow interop)