crawl-data

Crawl Data — Discover and Register Data Models

Scan a codebase directory for data model definitions and propose registry entries.

Arguments

• $1 — Path to scan (required). Absolute or relative path to the codebase to crawl.
• --domain <name> — Domain to assign discovered models to (optional, will ask if omitted).
• --write — Write proposed entries to registry immediately (default: preview only).

If no path is provided, ask the user which directory to scan.

Workflow

1. Discover Type-to-Folder Mapping

Read models/registry-mapping.yaml to find:

• The folder path for data_concept entries
• The folder path for data_aggregate entries
• The folder path for data_entity entries
• The _template.md in each folder for frontmatter structure

Never hardcode paths. Always derive from the YAML.

2. Scan for Data Model Definitions

Search the target directory for data model files. Use these detection patterns:

SQL Schema files:

• Glob: **/*.sql, **/migrations/**/*.sql, **/schema/**/*.sql
• Content match: CREATE TABLE, ALTER TABLE
• Extract: table name, column names/types, constraints, foreign keys

Prisma models:

• Glob: **/schema.prisma, **/*.prisma
• Content match: model <Name> {
• Extract: model name, fields with types, relations (@relation)

TypeORM / Sequelize / Drizzle (TypeScript ORMs):

• Grep for: @Entity(), @Table, Model.init, pgTable(, mysqlTable(
• Look in: **/models/**, **/entities/**, **/schema/**
• Extract: class/table name, decorated columns, relations

Pydantic / dataclass models (Python):

• Grep for: class.*BaseModel, @dataclass, class.*Model.*models.Model (Django)
• Look in: **/models/**, **/schemas/**, **/domain/**
• Extract: class name, field names/types, validators

TypeScript interfaces / types:

• Grep for: export interface, export type.*=.*{
• Look in: **/types/**, **/interfaces/**, **/models/**
• Extract: interface/type name, property names/types