task-scheduling

Task Scheduling

Decide when work runs and which worker runs it: fire jobs on a schedule (cron/delayed/recurring), hand each job to exactly one worker via a lease, and make sure it completes once despite crashes and retries. This sits on top of messaging-streaming queues — the queue is the transport; this skill adds the scheduling, leasing, priorities, and task-level idempotency. Getting it wrong shows up as jobs that never run, run twice (double charge, double email), or pile up until a worker fleet falls permanently behind.

When to reach for this

Work must run later (send a reminder in 24h), on a schedule (nightly rollups, hourly cron), or repeatedly (poll every 5 min); a slow operation is already off the request path (→ messaging-streaming) and now needs reliable allocation to a pool of workers; jobs need priorities (paid before free) or fairness (no single tenant starves others); or a job must complete exactly once even though the worker holding it can crash mid-flight.

When NOT to

The caller needs the result inline — that's a synchronous call, not a scheduled job. A single fire-and-forget async step with no schedule, priority, or exactly-once need — a plain queue + idempotent consumer (messaging-streaming) is simpler; don't add a scheduler on top. One periodic job on one box — OS cron is fine until you have multiple schedulers or need history and retries. A long-running multi-step saga with rollback — reach for a durable workflow engine instead of hand-rolling state across jobs. Don't stand up Airflow/Celery "because we'll have batch jobs eventually" (YAGNI): it's a stateful control plane to operate and monitor.

Clarify first

• Trigger type — scheduled (cron/at a time), delayed (run after N seconds), recurring (every N), or event-driven (a queue message arrives)? This decides whether a scheduler is even in scope.
• Exactly-once vs at-least-once — is a duplicate run harmful (money, email) or harmless (idempotent recompute)? Drives the leasing + dedup design.
• Latency budget vs throughput — must a delayed job fire within seconds of its time, or is "within a few minutes" fine? Tight timing is more expensive.
• Priority / fairness — do some jobs jump the line, and must one tenant or job class be prevented from starving the rest? (→ back-of-the-envelope for arrival vs. service rate.)
• Job duration & variance — seconds or hours? Sets the visibility-timeout / lease length and whether long jobs need heartbeats.
• Idempotency key — what identifies a task as the same task on retry? (The key contract is owned by api-design.)