backend-transactional-outbox

Backend Transactional Outbox

Purpose

Guarantee reliable event publication by writing events to an outbox table within the same database transaction as the business operation, then publishing them via a separate message relay process.

Agent Protocol

Trigger

Exact user phrases: "outbox", "transactional outbox", "outbox pattern", "reliable event publishing", "dual write", "CDC outbox", "message relay", "publish events", "exactly-once publish", "debezium outbox".

Input Context

• Database technology (PostgreSQL, MySQL, SQL Server, etc.).
• Message broker (Kafka, RabbitMQ, SQS, etc.).
• Whether dual-write (DB + broker) is a current problem.
• Current event publishing mechanism.

Output Artifact

Outbox implementation design as text. No file unless requested.

Response Format

Outbox table: {table schema}
Write strategy: {same transaction|CDC}
Relay: {polling|CDC|transaction log}
Delivery: {at-least-once|exactly-once}
Consumer dedup: {key and storage}

Completion Criteria

• Outbox table created in the same database as business data.
• Business operation and outbox insert are in the same DB transaction.
• Message relay process is separate from the application.
• Outbox records are deleted or marked processed after successful publish.
• Relay handles failures with retry and backoff.
• Consumers are idempotent (handle duplicate deliveries).
• Monitoring in place for outbox backlog.

Max Response Length

15 lines for design. 8 lines for table schema.

Decision Tree

Relay Strategy

What throughput do you need?
  ├── Low to moderate (<1000 events/sec), want simplicity
  │   └── Polling relay — periodic SQL query for unprocessed rows
  ├── High throughput (>1000 events/sec), need near real-time
  │   └── CDC relay (Debezium) — tail the WAL, no polling overhead
  └── Moderate throughput, want logical replication, no extra infra
      └── PostgreSQL LISTEN/NOTIFY + polling as fallback

When to Use Outbox Pattern