dq-rules

Data quality rules

A data quality rule (a "monitor") is a check attached to an existing DQ job (a dataset — a saved data-quality check on one database table). This skill covers authoring a custom rule (validate → create), inspecting it, and reading its per-run results. It does not cover creating the job itself, editing or deleting rules, or triggering job runs — a new rule is evaluated on the job's next (scheduled) run.

Rule tools: validate_data_quality_rule, create_data_quality_rule, get_data_quality_rule, get_data_quality_rule_results.

Hard rules

1. Validate before you create. Always call validate_data_quality_rule on the rule SQL before create_data_quality_rule. It checks the SQL against the source and returns valid: true/false plus a message, so a malformed rule (e.g. a bad SIMPLE_SQL/SQLG predicate) is caught up front. validate_data_quality_rule takes the raw SQL — the rule does not need to exist yet. If valid is false, fix the SQL and re-validate; do not create the rule.
   • create_data_quality_rule has a confirm checkpoint. Call it first with confirm omitted/false: it returns a preview (the composed rule and its SQL) and creates nothing. Show that preview to the user, then call again with confirm: true to actually create. The tool enforces this — it will not write on a confirm=false call.
2. validate_data_quality_rule needs discovery IDs. It requires edgeSiteId, connectionId and schemaName. Get them from prepare_create_data_quality_job for the target job — do not guess them. (create_data_quality_rule, get_data_quality_rule and get_data_quality_rule_results take only names/ids and need no discovery step.)
3. monitorType is FREEFORM_SQL or SIMPLE_SQL. FREEFORM_SQL is a full SQL query; SIMPLE_SQL is a single-column predicate. Nothing else is valid.
4. Always give the rule a meaningful name. monitorName is required and is how the rule is found and reported on later. Ask the user for a name; if they don't supply one, propose a clear, descriptive name (e.g. orders_amount_not_null) and confirm it before creating — do not invent an opaque name. Names allow only letters, digits, - and _.
5. For SIMPLE_SQL, ask which column the check targets and pass it as columnName. For FREEFORM_SQL the column(s) live inside the SQL, so columnName is not needed.
6. Rules require a PUSHDOWN job. If create_data_quality_rule returns an error mentioning the dataset is not PUSHDOWN (HTTP 422), rule creation is not allowed on that job — tell the user rather than retrying.
7. Read the status field in every response. Branch on success, validation_error, or error. For validate_data_quality_rule, status: success means validation ran — the verdict is the separate valid field.
8. Creating a rule does not run it. A new rule is only evaluated on the job's next run (runs happen via the job's schedule — this skill does not trigger them). Once a run has happened, use get_data_quality_rule_results to see how the rule did.