explore

Explore

Make sense of a warehouse or a local DuckDB database the way an analytics engineer does: rank what matters, drill selectively, and persist a draft map. This is the flagship, fully read-only skill. It absorbs profiling and relationship inference as capabilities; they are not separate skills.

How to drive it

Run the engine through the wrapper. It prints one sanitized JSON envelope and nothing else; read the envelope and decide the next step.

uv run "${CLAUDE_SKILL_DIR}/scripts/run.py" <subcommand> [flags]

dex runs its engine through uv, which is a prerequisite and is not installed by Claude Code. If the shell reports uv: command not found, stop and tell the user to install it (curl -LsSf https://astral.sh/uv/install.sh | sh, or brew install uv, or pipx install uv), then re-run. Never fall back to raw Python, pip, or a database CLI to do the work another way: the guardrails live in the engine, so any other path is unguarded.

If the user has no warehouse to point at and wants to see what dex does, demo generates one: a seeded local DuckDB warehouse plus the .dex/config.yml for it, with no credentials and no network, so every subcommand below then runs with no flags. It only ever creates, so it refuses rather than touch a file that already exists. Offer it rather than assuming it: a user who does have a warehouse wants that one read, not a fixture built beside it.

Subcommands, in the usual order:

1. connect test --path <file.duckdb> confirms a read-only connection and reports capabilities.
2. explore inventory --rank returns a ranked object summary (counts and sizes, never rows).
3. explore profile <objects> (space- or comma-separated) returns column profiles, PII flags recorded as (column, category, confidence) and never example values, plus candidate keys, the likely grain, and data-quality warnings (e.g. a non-unique id that will fan out on joins). A generic *_name flag's confidence is refined by value-shape evidence from the same scan, in both directions: person-shaped values corroborate it, a closed reference vocabulary or long labels de-rate it below the firewall's blocking threshold, and missing evidence changes nothing (the flag itself is never removed). Distinct counts are approximate for scale, but any column that looks unique within approximation noise is escalated to an exact COUNT(DISTINCT) (distinct_count_exact: true), so uniqueness and grain verdicts rest on proof; a ~ prefix in a warning marks a count that is still approximate. A requested object whose cached profile is still fresh (same connector, schema unchanged, within profile_freshness_hours, default 24) is served from the cache (cache_hit_count) instead of re-scanned, so profiling a table map just wrote costs nothing to spend; pass --refresh to force a re-scan when the source changed in a way the free metadata check cannot see.
4. explore relationships returns inferred and declared joins with confidences, plus notes explaining what the inference examined (so an empty list is meaningful). Add --verify to measure each inferred join with an aggregate overlap probe (orphan fraction, confidence adjusted). A declared join has two sources: a relationships test, and (with --use-project) an entity two semantic models share, which the layer states outright with the key named per model. declared_by on an edge names that entity, semantic_join_count says how many came that way, and the notes call out the ones name-based inference did not find, which is the interesting set: a semantic layer routinely joins columns that share no name at all.
5. explore map writes or updates the .dex/ cache and returns the map (--verify works here too). Alongside the counts, data.objects gives each top-ranked object its row count, detected grain, candidate key, notable columns (each carrying the role that earned it a place: grain, key, join, or a PII flag) and data-quality findings, and data.edges gives the join edges in the same shape explore relationships returns. With --use-project each object also carries semantic_models, the semantic models that sit on that relation, which is what separates a load-bearing table from a merely large one: empty means nothing in the layer reads it. Read that payload instead of chaining profile and relationships to re-derive it; go to those two when you need one object in full, or a value domain, which map never carries. It is budgeted: 25 objects by rank, 12 columns per object, 40 edges, 5 findings per object. Every cap binds in every mode and every elision is counted in notes and in an elided_* field, so an empty notes means nothing was cut. --detail widens the selection to every column and to objects that were inventoried but never profiled, and lifts no cap; it spends nothing, unlike --full. Past 50 objects it profiles only the top 25 by rank and says so in notes (with skipped_count); pass --full to profile everything. On a re-map, objects skipped this run keep their prior profiles (carried_forward_count), each stamped with its own profiled_at so staleness is visible instead of column detail silently vanishing. A selected object whose cached profile is still fresh (same connector, schema unchanged, profiled within profile_freshness_hours, default 24) is reused without a re-scan (cache_hit_count), so re-runs cost nothing to spend; pass --refresh to force a full re-profile when the source changed in a way the free metadata check cannot see (e.g. rows changed but the schema did not). explore relationships and the standalone explore profile reuse fresh profiles the same way.
6. explore diagram [--full] renders the cached map as a Mermaid ER diagram in data.mermaid. Free and connectionless (it reads the cache, never the warehouse), so it is safe to re-run while shaping the picture. Reproduce the string verbatim in a fenced ```mermaid block so the human can see it, and write it to a .mmd or a markdown file when they want one on disk: the engine deliberately writes no file. Never redraw or "tidy up" the diagram by hand. The glyphs are claims the engine derived from evidence, and a plausible-looking cardinality you supplied is exactly the overclaim this command exists to prevent: declared joins are solid, inferred dotted, and an unverified inference never says "exactly one". A solid line labelled with a semantic entity is a join the semantic layer declares; look the entity up with explore semantic list. Read notes before presenting it, since it states any object or column that was left out; --full widens from the default (profiled, joined objects and their grain, key, join, and PII columns) to everything eligible.
7. explore query "<SELECT ...>" ["<SELECT ...>" ...] answers ad-hoc questions the fixed commands don't cover: you write the SQL, the engine's query firewall refuses or bounds it. Pass a statement per argument, or --sql-file <path> for a longer list, and ask a whole chain of questions in one call rather than one call each; each statement is judged and answered on its own, so a refusal on one does not cost you the others, and data.results carries one entry per statement. A table you have not profiled, including a model you just built, is profiled for you and the statement then runs, so probing something new is one call rather than three; the envelope says what it profiled, and on a metered connector that profile is priced into the same confirmation as the statements. Results come back row-major and capped; a refusal names the offending column and the fix, so one rewrite is enough. Read ${CLAUDE_SKILL_DIR}/references/probe-playbook.md before writing a probe: it maps common questions to effective probe shapes.
8. explore cluster <object> [--features a,b,c] [-k N] runs k-means over a bounded sample of the object's numeric columns and returns the segment structure: per-cluster sizes and fractions, centroids (each coordinate is a cluster's mean of that feature, an aggregate), the silhouette score, and, when -k is omitted, the k it picked plus the silhouette sweep it chose from. Requires the .dex/ cache (run map/profile first) so features can be auto-selected from profiled numeric, non-PII, non-key columns; pass --features to choose them yourself (naming a PII column, or a key, opts it in deliberately, and only its mean is ever reported). A key is never a feature: its mean is meaningless, and a fact table is mostly keys plus a handful of measures, so clustering on them just partitions surrogate ranges. Keys are the unique columns, the columns that join out (from the joins map inferred), and the columns named like one; prefer map over a bare profile here, because without inferred joins a foreign key is caught only if its name gives it away. The notes name every excluded column, so check them before trusting a result. Two things the silhouette alone will not tell you, both of which the notes will. A cluster holding under 1% of the sample is an outlier pocket, not a segment, and it pushes the score up precisely because it sits so far out: report that as outlier detection, or re-run with -k to split the bulk. And on connectors that cannot seed a sample the draw changes per run, so two runs can disagree on k; the envelope's sample_repeatable says which case you are in, and comparing runs across different draws is meaningless. Only aggregates cross the boundary: the sample rows are clustered in-process and never enter context. On a metered connector it takes the same cost handshake as the scanning commands below (only the feature columns are scanned, and a dialect-aware sample clause reads a fraction), so surface the estimate and get a budget first. Needs the [cluster] extra (scikit-learn); the wrapper installs it automatically for this subcommand.
9. explore semantic list|values|query reach the semantic layer: the metrics an author defined, and the semantic models, measures, dimensions and entities they are built out of. Distinct from the warehouse commands above, and from the top-level semantic group, which authors the layer where this queries it.