submitqueue AGENTS.md

SubmitQueue Repository Guide

Key Concepts

SubmitQueue is a distributed system for managing code submission workflows. It follows clean architecture with interface-driven extensibility.

Immutability and Eventual Consistency:

1. Immutable entities — once created, don't modify in place. Create new versions with updated fields.
2. Eventual consistency — handle stale reads, idempotent operations, and convergence over time.
3. Event sourcing — store events (what happened) rather than just current state for critical changes.
4. Optimistic locking — use version numbers instead of pessimistic locks. Avoid transactions; prefer optimistic concurrency and retries. Version arithmetic lives in the controller, not the storage layer. Update methods take both oldVersion (the where-clause guard) and newVersion (the value to write); the store performs a pure conditional write. Controllers compute newVersion = oldVersion + 1, call the store, and only assign entity.Version = newVersion after the call succeeds. Pre-incrementing in memory before the call is a bug pattern — on error the in-memory version drifts ahead of the database. See submitqueue/extension/storage/README.md.
5. Idempotency keys — include unique request IDs, check for duplicates before executing.
6. Persist before you publish — when a message hands work to a later stage, write the state first and publish only once the write has succeeded. Publishing first races the consumer against the write: it can load an entity that was never recorded, or a version older than the one the message describes, and then build on an assumption that was never true. Ordered write-then-publish, a failed publish leaves the state durable and the retry re-publishes; the reverse leaves a message describing a state nothing wrote. The exception is a message that depends on nothing the write does — a status log recording a transition, or an idempotent nudge whose consumer re-derives everything from current state. Those publish first on purpose, so that a failure leaves nothing written and the retry redoes the whole step rather than stranding a durable state change with no way to announce it; see submitqueue/orchestrator/controller/speculate/finalize.go and .../buildsignal/buildsignal.go for the reasoning at those two sites.