kafka-consumer-lag

Kafka Consumer Lag Analysis

Analyses consumer group lag across all groups and diagnoses potential causes. Consumer lag is the most commonly monitored Kafka metric and the first thing engineers check during incidents.

Target environment: $ARGUMENTS

Workflow

Copy this checklist and track your progress:

Lag Analysis Progress:
- [ ] Step 1: Fetch all consumer groups
- [ ] Step 2: Identify problematic groups
- [ ] Step 3: Diagnose root causes
- [ ] Step 4: Generate report

1. Fetch all consumer groups with state and lag info
2. Identify problematic groups (high lag, idle, rebalancing)
3. Diagnose root causes using partition and throughput data
4. Report findings with per-group remediation steps

Step 1: Fetch Consumer Groups

Use the Lenses MCP list_consumer_groups tool to get all consumer groups with:

• Group state (Stable, Rebalancing, Empty, Dead)
• Lag per topic-partition
• Active member count
• Coordinator info

For topic-specific analysis, use list_consumer_groups_by_topic to narrow the scope.

Expected output: List of all consumer groups with state, lag and member count.

Validation: If no consumer groups are returned, report this finding and stop - the cluster may have no active consumers.

Step 2: Identify Problematic Groups

Flag consumer groups in these categories:

Critical

• Groups in Dead or Empty state with committed offsets (consumer has stopped)
• Groups with lag growing over time (compare current lag against partition end offsets using get_topic_partitions)

Warning

• Groups in Rebalancing state (consumption paused)
• Groups with uneven partition assignment (some consumers have significantly more partitions than others)
• Groups with no committed offsets (consumer may never have started successfully)

Suggestion

• Groups with very high member count relative to partition count (idle consumers)
• Groups consuming from topics with no recent messages (use get_dataset_message_metrics to check producer throughput)