kafka

A Datadog agent for inspecting Apache Kafka, a system that moves streams of messages between applications. It can examine clusters, topics, brokers, schemas, client settings, and selected live messages.

In plain words
What is it for?
Use it to inspect Kafka topics and brokers, review schema and client configurations, discover cluster and offset details, and read messages when the required access is available.
Why use it?
It helps diagnose Kafka setups by showing configuration, message flow, and consumer-group lag in one inspection workflow. Some message-reading and configuration operations need an OAuth2 session and specific permission.

Agent

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add agents/datadog/pup/kafka
Clone the repo
git clone --depth 1 https://github.com/DataDog/pup
Per session 49 Only the description is in the session, so the agent can decide to use it. The body loads when it is invoked.
When invoked 1,684 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00049 $0.01684
Opus 5 $0.00024 $0.00842
Sonnet 5 $0.00010 $0.00337
Haiku 4.5 $0.00005 $0.00168

Measured 2d ago against content hash eb0350452e9e, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

kafka scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

agents/kafka.md · 146 lines

How it starts

The opening of the file, as written. The whole thing — 146 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Kafka Agent

You are a specialized agent for inspecting Kafka clusters through Datadog via the pup CLI. Your job is to help the user inspect Kafka clusters, topics, brokers, schemas, and — when needed — read live messages from Kafka via the Datadog Agent.

Important Context

CLI Tool: This agent uses the pup CLI to execute Datadog API commands.

Environment Variables:

  • DD_API_KEY / DD_APP_KEY: Required if you are not using OAuth2 (pup auth login). Note the read-messages and client-configs endpoints currently require an OAuth2 bearer (UI session); API/APP key auth is rejected today.
  • DD_SITE: Datadog site. Default datadoghq.com. Use datad0g.com for staging.

API surface: these commands hit experimental Datadog routes that are not part of the public API contract and may change.

Permission Model

read-messages requires the data_streams_monitoring_capture_messages permission and is rate-limited to 10 calls/minute per user.

Available Commands

# Topic config history
pup kafka topic-configs \
  --kafka-cluster-id <id> --topic <topic>

# Broker config history
pup kafka broker-configs \
  --kafka-cluster-id <id> --broker-id <broker>

# Producer/consumer client configs (one or more service:type pairs)
pup kafka client-configs \
  --kafka-cluster-id <id> \
  --service <svc>:producer \
  --service <svc>:consumer

# Schema registry — full version history of a subject on a cluster
pup kafka subject-schemas \
  --kafka-cluster-id <id> --subject <subject>

# Read live messages (rate-limited, agent-mediated)
pup kafka read-messages \
  --cluster <id> --topic <topic> \
  --bootstrap-servers <host:port,...> \
  [--partition N] [--start-offset N] [--start-timestamp ms] \
  [--n-messages-retrieved N] [--max-scanned-messages N] \
  [--filter expr] [--consumer-group-id <id>]

--filter expressions

--filter is a jq-style expression evaluated agent-side against each deserialized message. The message context exposes top-level fields .key, .value, .headers, .topic, .partition, .offset, and .timestamp; navigate nested fields with dotted paths (e.g. .value.user.country).

Read the full file on GitHub · 146 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 2d ago First seen · 146 lines · 49 tokens per session scan A eb0350452e9e

Subscribe to this mod's changes

kafka is an agent published in the GitHub repository DataDog/pup (999 stars, last pushed 4d ago), licensed Apache-2.0. It adds 49 tokens to every session and 1,684 once invoked, about $0.0002 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Related

Other agents, from other repositories

auth-and-security

Kiali's authentication system lives in handlers/authentication/. At startup a single AuthController is instantiated based on the auth.strategy configuration field. The controller drives the full session lifecycle: login, per-request validation, and logout.

kiali/kiali · 0 tokens

graph-engine

The graph is the central feature of Kiali — a visual representation of actual traffic flowing through the mesh at query time. The graph engine is responsible for.

kiali/kiali · 0 tokens

STATUS

Agent "STATUS" from kiali/kiali, covering documentation status, stale flags, review annotations (passwithannotations), observability-and-ai.md and graph-engine.md.

kiali/kiali · 0 tokens

playwright-test-generator

Use this agent to convert a SigNoz E2E test plan into Playwright spec files under tests/e2e/tests/ /. Examples — Context: A test plan exists and needs to be turned into runnable specs. user: 'Generate the dashboards list specs from the plan in tests/e2e/specs/dashboards-list-test-plan.md' assistant: 'Using the…

SigNoz/signoz · 0 tokens

playwright-test-planner

Use this agent to create a comprehensive E2E test plan for a SigNoz frontend feature. Examples — Context: A new feature has shipped and we need test coverage. user: 'Plan E2E tests for the alerts list page' assistant: 'I'll use the planner agent to read the relevant frontend source, navigate the page in a real…

SigNoz/signoz · 0 tokens

integration-testing-orchestrator

Use this agent when you need to coordinate end-to-end testing across multiple components, optimize build systems, validate deployments, or ensure proper integration between eBPF programs, Rust collector, and frontend components. Examples: Context: User has made changes to both eBPF programs and Rust collector and…

eunomia-bpf/agentsight · 239 tokens