job-debug

Job / CronJob Failure Diagnosis

When a Job has failed, is stuck, or a CronJob is not triggering as expected, follow this flow to identify the root cause.

Scope: This skill is for diagnosis only. Once you identify the root cause, report it to the user and stop. Do NOT attempt to delete, restart, or modify Jobs or CronJobs — that should be left to the user.

Diagnostic Flow

1. Check Job status

kubectl get jobs -n <ns>

Note the COMPLETIONS (succeeded/total) and DURATION columns. For a specific Job:

kubectl describe job <job-name> -n <ns>

Focus on:

• Conditions — look for Complete (success) or Failed (failure) with the reason
• Events — look for BackoffLimitExceeded, DeadlineExceeded, or pod creation errors
• Pods Statuses — counts of Succeeded, Failed, and Active pods

2. Check the Job's pods

kubectl get pods -n <ns> -l job-name=<job-name> --sort-by='.metadata.creationTimestamp'

Note pod statuses. For failed pods, check logs:

kubectl logs <pod-name> -n <ns>

If the pod was terminated, check the exit code:

kubectl get pod <pod-name> -n <ns> -o jsonpath='{.status.containerStatuses[*].state.terminated}'

3. For CronJobs — check schedule and history

kubectl get cronjobs -n <ns>

Note the SCHEDULE, SUSPEND, ACTIVE, and LAST SCHEDULE columns.

For details:

kubectl describe cronjob <cronjob-name> -n <ns>

Focus on:

• Schedule — the cron expression
• Suspend — if True, the CronJob will not create new Jobs
• Last Schedule Time — when the last Job was triggered
• Active Jobs — currently running Jobs
• Events — look for creation events or errors

List Jobs created by the CronJob:

kubectl get jobs -n <ns> -l job-name --sort-by='.metadata.creationTimestamp' | grep <cronjob-name>

4. Match patterns and conclude

---

BackoffLimitExceeded — Too many pod failures

The Job's pods have failed more times than the backoffLimit (default: 6). The Job is marked as Failed.