validate-taxonomy

Validate Taxonomy

Inspect a taxonomy file for common issues before database load. Run validation checks and report findings.

When to use

• "Validate this CSV before I load it"
• "Check the taxonomy for errors"
• "Does this have any duplicate codes or missing values?"

Inputs to gather

• Taxonomy file path: CSV or JSON.
• Expected columns: Primary key column (usually code), required fields.
• Hierarchical? If yes, which column is parent_code?

Procedure

1. Read the taxonomy file (CSV or JSON).
2. Run validation checks:
   • Primary key uniqueness: All codes are unique; no nulls in the PK column.
   • Required-field completeness: No nulls in required columns (name, label, description).
   • Code format consistency: Infer dominant pattern (snake_case, UPPERCASE, alphanumeric-only, etc.); warn on outliers.
   • Code length budgets: codes ≤ 32 chars, labels ≤ 255 chars.
   • Referential integrity (hierarchical only): Every parent_code resolves to a code in the table; no NULL parent_code unless it's a root.
   • Cycle detection (hierarchical only): DFS from each root; flag any backward edges.
   • Duplicate labels: Warn (sometimes legitimate — e.g., multiple codes for the same name); do not error.
3. Write report to state/validations/<timestamp>-<filename>.txt with:
   • Summary: pass/fail, count of errors/warnings.
   • Per-check: pass/fail, details (row numbers, example values).
   • Exit code 0 if no errors; 1 if errors found.
4. Print the report to stdout and return the exit code.

Output / side effects

• Validation report logged to state/validations/<timestamp>-<filename>.txt.
• Exit code indicates success/failure.

Safety / constraints

• Non-destructive: Validation does not modify the file.
• Warnings vs errors: Code format outliers and duplicate labels are warnings; PK uniqueness and referential integrity are errors.