loom-data-validation

Data Validation

Overview

Validate untrusted data at trust boundaries before it flows into your system. This skill covers schema libraries (Zod/Pydantic/Joi/JSON Schema), coercion pitfalls, context-dependent output encoding, injection/XSS/DoS defenses, and pipeline/ML feature validation.

Core principles (read first)

• Parse, don't validate. A validator that returns bool throws away work — the caller re-parses or trusts blindly. Return a typed value (Result<User>, User | errors) so downstream code cannot receive unvalidated data. Schema libraries (Zod .parse, Pydantic .model_validate) do this by construction.
• Validate at the boundary, once, then trust the typed value inward. Boundaries: HTTP handlers, queue consumers, file/CLI parsers, pipeline ingestion, cross-service calls.
• Server-side is authoritative; client-side validation is UX only. Never rely on it for security — attackers bypass the client entirely.
• Allowlist > denylist. Enumerate what's permitted (enum, char classes, known hosts). Denylists (blocking <script>, ../, ') are always incomplete — encodings, Unicode, and case defeat them.
• Canonicalize before validating. Normalize Unicode (NFC), lowercase host, resolve ./.. in paths, decode percent-encoding — then check. Validating raw input lets %2e%2e%2f or ． (fullwidth) slip past.
• Encoding ≠ validation. Validation decides accept/reject; encoding makes a value safe for a specific sink (HTML vs attribute vs JS vs URL vs shell vs SQL). A value can be valid and still need encoding at every sink.
• Limits are validation. Cap length, array size, object depth, and total payload bytes to stop DoS (JSON bombs, deeply nested payloads, ReDoS amplification).

Zod (TypeScript)

safeParse returns a discriminated result (no throw); parse throws ZodError. Prefer safeParse at boundaries.

import { z } from "zod";

const CreateUser = z
  .object({
    email: z.string().trim().toLowerCase().email().max(255),
    password: z.string().min(12).max(128)
      .regex(/[a-z]/).regex(/[A-Z]/).regex(/\d/).regex(/[^A-Za-z0-9]/),
    role: z.enum(["user", "admin", "moderator"]).default("user"),
    tags: z.array(z.string().max(50)).max(10).default([]),
    age: z.number().int().min(13).max(150).optional(),
  })
  .strict();            // reject unknown keys → blocks mass-assignment/overposting

type CreateUserIn = z.input<typeof CreateUser>;   // pre-transform
type CreateUserOut = z.output<typeof CreateUser>; // post-transform (use this downstream)

const parsed = CreateUser.safeParse(req.body);
if (!parsed.success) {
  const errors = parsed.error.issues.map((e) => ({ field: e.path.join("."), message: e.message }));
  return res.status(422).json({ errors });
}
const user = parsed.data; // fully typed + validated