test-data-generator

When to use

Invoke this skill when:

• A new integration test needs seeded rows in staging.
• An e2e test needs JSON fixtures matching a schema.
• A developer wants bulk data for a local DB (1k–100k rows).
• CSV import flows need test payloads at multiple sizes.

Do NOT use when: the data must be genuine PII from production (never; mask at source instead); the schema does not exist (write it first).

Inputs

• schema (required): Path to JSON Schema, OpenAPI component, or SQL DDL.
• count (required): Number of records per entity.
• format (required): sql, json, or csv.
• seed (optional): Integer seed for faker. Default 42.
• locale (optional): Faker locale, e.g. en, de, ja. Default en.
• relations (optional): Explicit FK order, e.g. ["users", "orders", "order_items"].
• overrides (optional): Per-field generators, e.g. { "users.email": "{{internet.email}}" }.

Outputs

• Generated data file(s):
  • sql: one .sql file per table with INSERT statements wrapped in a transaction.
  • json: one .json file per entity (array of records).
  • csv: one .csv per entity with header row.
• A manifest.json recording the seed, locale, counts, and hash of the output for reproducibility.

Tool dependencies

• Read, Write, Glob, Grep (always).
• JS: @faker-js/faker (v8+), ajv for post-generation validation.
• Python: Faker (pip install faker), jsonschema.
• Optional mimesis (Python) for faster bulk generation.

Procedure

1. Parse the schema.
   • JSON Schema / OpenAPI: use $ref resolution; collect required, enum, format, pattern, minLength/maxLength, minimum/maximum, uniqueItems.
   • SQL DDL: extract CREATE TABLE, column types, NOT NULL, UNIQUE, CHECK, REFERENCES (FK), defaults.
2. Topologically sort tables by FK so parents are generated before children.
3. Set the seed. faker.seed(seed) (Python) or faker.seed(seed) (JS). Document in manifest.
4. Pick generators. Map column name + type + format to a faker call. Apply overrides last.
5. Honor constraints.
   • unique: generate into a Set and retry on collision; if after 10× count tries still colliding, widen the pool (e.g. append an index suffix).
   • enum: pick from the enum list.
   • min/max: clamp the faker output.
   • pattern: regenerate until matching; if pattern is too narrow, use a randexp-style generator.
   • foreign key: pick a random parent row id from already-generated parents.
6. Validate. Run each record through ajv (JSON/OpenAPI) or a DDL-derived check (SQL) before emitting. Fail loudly on any validation error.
7. Emit output.
   • SQL: BEGIN; INSERT INTO ...; COMMIT; per file; chunk inserts at 1,000 rows per INSERT for speed.
   • JSON: JSON.stringify(data, null, 2); keep under 50 MB per file (split if larger).
   • CSV: RFC 4180 quoting; UTF-8 with BOM only if the target tool needs it.
8. Write manifest.

   { "seed": 42, "locale": "en", "counts": { "users": 1000, "orders": 5000 }, "hash": "sha256:..." }
   

9. Report. List files, row counts, any constraints that were widened (e.g. unique pool exhaustion).