design-serialization-schema

Design Serialization Schema

Create well-versioned serialization schemas that evolve gracefully without breaking consumers.

When to Use

• Defining a new API contract or data interchange format
• Adding fields to an existing schema without breaking consumers
• Migrating between schema versions
• Choosing between schema systems (JSON Schema, Protobuf, Avro)
• Documenting data validation rules for automated enforcement

Inputs

• Required: Data model (entity relationships, field types, constraints)
• Required: Compatibility requirements (who consumes this data, how long must old formats be readable)
• Optional: Existing schema to evolve
• Optional: Performance requirements (validation speed, schema registry integration)
• Optional: Target serialization format (JSON, binary, columnar)

Procedure

Step 1: Choose a Schema System

System	Format	Strengths	Best For
JSON Schema	JSON	Widely supported, flexible validation	REST APIs, config validation
Protocol Buffers	Binary	Compact, fast, strong typing, built-in evolution	gRPC, microservices
Apache Avro	Binary/JSON	Schema in data, excellent evolution support	Kafka, data pipelines
XML Schema (XSD)	XML	Comprehensive typing, namespace support	Enterprise/legacy SOAP
TypeBox/Zod	TypeScript	Type inference, runtime validation	TypeScript APIs

Got: Schema system selected based on ecosystem, performance needs, and evolution requirements. If fail: If uncertain, start with JSON Schema — it has the broadest tooling support and can be layered onto existing JSON APIs.

Step 2: Design the Core Schema

JSON Schema example:

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://example.com/schemas/measurement/v1",
  "title": "Measurement",
  "description": "A sensor measurement reading",
  "type": "object",
  "required": ["sensor_id", "value", "unit", "timestamp"],
  "properties": {
    "sensor_id": {
      "type": "string",
      "pattern": "^[a-z]+-[0-9]+$",
      "description": "Unique sensor identifier (lowercase-digits format)"
    },
    "value": {
      "type": "number",
      "description": "Measured value"
    },
    "unit": {
      "type": "string",
      "enum": ["celsius", "fahrenheit", "kelvin", "percent", "ppm"],
      "description": "Unit of measurement"
    },
    "timestamp": {
      "type": "string",
      "format": "date-time",
      "description": "ISO 8601 timestamp with timezone"
    },
    "metadata": {
      "type": "object",
      "additionalProperties": true,
      "description": "Optional key-value metadata"
    }
  },
  "additionalProperties": false
}