lab

Lab - Lab Environment Setup Skill

Arguments: $ARGUMENTS

---

Design Principles

1. Technology isolation: When practicing Kafka, do not spin up ES. Each lab starts and stops independently.
2. Generated to match learning content: Instead of a pre-built generic compose, generate a configuration that matches the current learning topic.
3. Monitoring included: Every lab includes monitoring appropriate for that technology by default.
4. Incremental expansion: Start with a basic configuration and add load testing/APM as needed.

---

Directory Structure

labs/
├── kafka/
│   ├── docker-compose.yml
│   ├── docker-compose.monitoring.yml   (optional)
│   ├── docker-compose.loadtest.yml     (optional)
│   └── apps/
│
├── elasticsearch/
│   ├── docker-compose.yml
│   ├── docker-compose.apm.yml          (optional)
│   └── apps/
│
├── database/
│   ├── docker-compose.yml
│   ├── docker-compose.monitoring.yml   (optional)
│   ├── data/
│   └── queries/
│
├── redis/
│   ├── docker-compose.yml
│   ├── docker-compose.monitoring.yml   (optional)
│   └── apps/
│
└── shared/
    ├── grafana/
    │   └── provisioning/
    ├── prometheus/
    │   └── prometheus.yml
    └── loadtest/
        └── k6/

---

Technology-Specific Configuration Guide

Kafka Lab

Basic Configuration (docker-compose.yml)

# Components:
# - Zookeeper (1 node) or KRaft mode (without Zookeeper)
# - Kafka Broker x3 (cluster)
# - Kafka UI (for viewing topics/messages/consumer groups)
#
# Ports:
# - 9092, 9093, 9094: Kafka brokers
# - 8080: Kafka UI
#
# Network: kafka-net (bridge)

Things to verify when generating:

• KRaft vs Zookeeper: KRaft mode is recommended for Kafka 3.x and above (no Zookeeper needed)
• Memory: Limit KAFKA_HEAP_OPTS to 512MB~1GB per broker (since it runs locally)
• Volumes: Use named volumes for data persistence, or tmpfs if persistence is not needed

Adding Monitoring (docker-compose.monitoring.yml)

# Additional components:
# - JMX Exporter (attached as agent to each broker)
# - Prometheus (metrics collection)
# - Grafana (dashboards)
#   - Preset dashboards: Kafka Broker Metrics, Consumer Lag
#
# Start: docker compose -f docker-compose.yml -f docker-compose.monitoring.yml up -d