backend-background-jobs

Backend Background Jobs

Purpose

Design reliable background job processing with queue topology, retry, concurrency, and monitoring. Every job must be idempotent, retryable, observable, and gracefully handled on failure.

Agent Protocol

Trigger

Exact user phrases: "background job", "task queue", "worker", "job processing", "cron job", "scheduled task", "async task", "job retry", "queue worker", "Sidekiq", "Celery", "Bull", "delayed job", "job priority", "job scheduling".

Input Context

• Job types (fire-and-forget, delayed, scheduled, chained).
• Queue backend (Redis, SQS, database table, RabbitMQ).
• Job duration (affects concurrency and timeout).
• Retry requirements (max retries, retry interval, irrecoverable failure definition).

Output Artifact

Job and worker design as formatted text.

Response Format

// Job contract (interface/type)
// Worker implementation outline

# Queue topology config
# Retry policy
# Concurrency rules

No preamble. No postamble. No explanations. No filler/hedging/transitions. Compress output.

Completion Criteria

• Job defined with idempotency key and full payload schema
• Retry strategy with exponential backoff and jitter configured
• Queue topology defined (default, priority, dead letter, scheduling queues)
• Worker concurrency and graceful shutdown configured
• Monitoring hooks for success/failure/metrics
• Scheduled/cron jobs defined with timezone and error notification

Architecture Decision Trees

Queue Backend Selection

What infrastructure do you already have?
├── Redis already in stack?
│   ├── Jobs < 100KB, sub-minute duration?
│   │   ├── Yes → Bull/BullMQ (built-in priority, delay, DLQ)
│   │   └── No → Consider RabbitMQ for complex routing
│   └── Need persistence and high throughput?
│       └── Bull with Redis Cluster or SQS
├── AWS-native?
│   ├── Jobs fit in 256KB? → SQS (fully managed, unlimited throughput)
│   ├── Need FIFO ordering? → SQS FIFO (exactly-once, 300 TPS)
│   └── Need long-running jobs? → SQS + Step Functions
├── Need transactional consistency with DB?
│   └── Database-backed queue (simplest setup, no infra)
└── Need complex routing (topic exchanges, headers)?
    └── RabbitMQ