data-quality-checker

Overview

Detect common data quality issues in market analysis documents before publication. The checker validates five categories: price scale consistency, instrument notation, date/weekday accuracy, allocation totals, and unit usage. All findings are advisory -- they flag potential issues for human review rather than blocking publication.

When to Use

• Before publishing a weekly strategy blog or market analysis report
• After generating automated market summaries
• When reviewing translated documents (English/Japanese) for data accuracy
• When combining data from multiple sources (FRED, FMP, FINVIZ) into one report
• As a pre-flight check for any document containing financial data

Prerequisites

• Python 3.9+
• No external API keys required
• No third-party Python packages required (uses only standard library)

Workflow

Step 1: Receive Input Document

Accept the target markdown file path and optional parameters:

• --file: Path to the markdown document to validate (required)
• --checks: Comma-separated list of checks to run (optional; default: all)
• --as-of: Reference date for year inference in YYYY-MM-DD format (optional)
• --output-dir: Directory for report output (optional; default: reports/)

Step 2: Execute Validation Script

Run the data quality checker script:

python3 skills/data-quality-checker/scripts/check_data_quality.py \
  --file path/to/document.md \
  --output-dir reports/

To run specific checks only:

python3 skills/data-quality-checker/scripts/check_data_quality.py \
  --file path/to/document.md \
  --checks price_scale,dates,allocations

To provide a reference date for year inference (useful for documents without explicit year in dates):

python3 skills/data-quality-checker/scripts/check_data_quality.py \
  --file path/to/document.md \
  --as-of 2026-02-28

Step 3: Load Reference Standards

Read the relevant reference documents to contextualize findings:

• references/instrument_notation_standard.md -- Standard ticker notation, digit-count hints, and naming conventions for each instrument class
• references/common_data_errors.md -- Catalog of frequently observed errors including FRED data delays, ETF/futures scale confusion, holiday oversights, allocation total pitfalls, and unit confusion patterns