schema-null-defaults

Handle NULLs and Default Values

Handle NULL values and set default values for missing fields in materializations.

Docs: https://docs.estuary.dev/concepts/schemas/#default-annotations

When to Use

• Need to provide default values for missing/optional fields
• Handling nullable primary key errors in materializations (cannot materialize collection with nullable key field unless it has a default value annotation)
• Ensuring consistent values when source data has gaps

Prerequisites

• Existing collection to modify
• flowctl authenticated

Key Concepts

• default applies only to missing fields, not to explicit null values. A field present with null stays null.
• default is read by materializations only — captures and derivations ignore it. Add defaults to the readSchema so auto-discover (which only touches writeSchema/connector-schema) can't overwrite them.

Procedure

Step 1: Pull Collection Specs

flowctl catalog pull-specs --name <tenant>/<collection-name>

Step 2: Identify the Read Schema

Find the read schema file (e.g., collection.read.schema.yaml).

Typical structure for collections using schema inference:

allOf:
  - $ref: flow://relaxed-write-schema
  - $ref: flow://inferred-schema

Step 3: Add Default Values

Add default annotations at the top level of the readSchema, alongside allOf:

allOf:
  - $ref: flow://relaxed-write-schema
  - $ref: flow://inferred-schema
properties:
  status:
    type: string
    default: "pending"
  count:
    type: integer
    default: 0
  is_active:
    type: boolean
    default: true

The properties block with defaults sits alongside the allOf, not inside it.

Step 4: Publish Changes

flowctl catalog publish --source flow.yaml

Common Patterns

Default for Optional String Field

allOf:
  - $ref: flow://relaxed-write-schema
  - $ref: flow://inferred-schema
properties:
  category:
    type:
      - string
      - "null"
    default: "uncategorized"