migrate

GreyCat Migration & Database Management

Purpose: Schema changes, data transformations, bulk import/export, storage maintenance.

⚠ These ops are sequential, interactive, and destructive — do them one at a time, never in parallel, and back up before anything risky.

---

Step 1: Choose Operation (AskUserQuestion)

• A. Schema Evolution — add/remove fields, detect breaking changes
• B. Data Migration — transform existing data
• C. Import/Export — bulk CSV/JSON ops
• D. Storage Health — defrag, backup, diagnostics

---

Operation A: Schema Evolution

A.1 Scan models

find src -name "*.gcl" -not -name "*_api.gcl" -not -name "*_reader.gcl" -not -name "*_writer.gcl"

Per file: extract type Name { ... } definitions + field lists.

A.2 Ask change type (AskUserQuestion)

Add field · Remove field · Change field type · Rename field (requires migration).

A.3 Add field — safety analysis

The runtime diffs the new program against the stored ABI on rebuild and auto-migrates compatible changes:

  • Add a NULLABLE attribute            → auto-migrates (existing instances read it as null).
  • Add a NON-NULL attribute WITH a default → auto-migrates (existing instances get the default).
  • Add a NON-NULL attribute WITHOUT a default → load FAILS — provide a default or write a migration.
  • Remove an attribute → auto (dropped on next save). Rename → NOT auto (looks like remove + add).

Options:

• A) Nullable (Recommended): field: T?; — no data touched, safe
• B) Non-null with default: field: T = <default>; — auto-migrates existing instances to the default
• C) Nullable → backfill → tighten: for a computed (non-constant) backfill, safe for production
• D) Dev reset: rm -rf gcdata (destructive, dev only)
• E) Cancel

Option C (computed backfill) — generate migration:

// src/migration/migrate_YYYYMMDD_<desc>.gcl
fn migrate_add_device_priority() {
  info("Start: add Device.priority");
  var count = 0; var errors = 0;
  for (id: int, device in devices_by_id) {
    try {
      device->priority = 1;  // user-provided default
      count = count + 1;
      if (count % 100 == 0) info("Migrated ${count}...");
    } catch (ex) { error("Failed ${id}: ${ex}"); errors = errors + 1; }
  }
  info("Done: ${count} updated, ${errors} errors");
}

Then: greycat run migrate_add_device_priority → set type non-nullable → greycat lint.