kafka-topic-audit

Kafka Topic Configuration Audit

Audits all topic configurations against production best practices. Misconfigured topics are the #1 cause of Kafka data loss - engineers create topics and forget to tune them.

Target environment: $ARGUMENTS

Workflow

Copy this checklist and track your progress:

Audit Progress:
- [ ] Step 1: Check environment health
- [ ] Step 2: Fetch all topics
- [ ] Step 3: Audit configurations against best practices
- [ ] Step 4: Check metadata completeness
- [ ] Step 5: Detect orphaned topics
- [ ] Step 6: Run consistency checks
- [ ] Step 7: Generate report

1. Check environment health for a high-level summary
2. Fetch all topics and their configurations
3. Audit each topic against best practices (see references/audit-rules.md)
4. Cross-reference metadata for completeness
5. Detect orphaned topics with no consumers
6. Report findings with prioritised recommendations

Step 1: Environment Overview

Use the Lenses MCP check_environment_health tool to get a quick summary:

• Broker count, topic count, consumer count, connector count
• Any existing issues flagged by Lenses

Expected output: Environment health summary with broker, topic and consumer counts.

Validation: If the environment is unhealthy or unreachable, stop and report the connection issue before proceeding.

Step 2: Fetch All Topics

Use the Lenses MCP list_topics tool to retrieve all topics with their configurations in one call.

For topics that need deeper inspection, use:

• get_topic for detailed config including partitions and consumers
• get_topic_broker_configs for broker-level config overrides
• get_topic_partitions for partition-level message counts and bytes

Expected output: Full list of topics with their configurations. If zero topics are returned, report this and stop.

Step 3: Audit Configurations

For each topic, check against the thresholds in references/audit-rules.md:

• Replication factor - RF=1 is critical, RF=2 is a warning in production
• Retention policies - unbounded growth, too short or excessively long
• Partition count - single-partition bottlenecks or excessive partitions
• Compaction settings - compact without keys, delete for state topics
• Naming conventions - must follow {domain}.{entity}.{event} pattern