What is kafka-python-client?

A project scaffold for Python applications that send messages to and read messages from Apache Kafka, including message schemas and tests. It uses confluent-kafka-python and connects to a Schema Registry, which stores the expected structure of messages.

Use it to generate a Python Kafka producer and consumer project after discovering the target topic, partitions, and registered JSON Schema from a connected Kafka system.

It avoids starting a Kafka client project from scratch and includes setup for safer message delivery, clean shutdowns, and testing.

kafka-python-client — Skill for Claude Code and Codex by lensesio: A project scaffold for Python applications that send messages…

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods

npx agentmods add skills/lensesio/agentic-engineering-for-apache-kafka/kafka-python-client

Any agent

npx skills add lensesio/agentic-engineering-for-apache-kafka --skill kafka-python-client

Clone the repo

git clone --depth 1 https://github.com/lensesio/agentic-engineering-for-apache-kafka

Made for: Claude Code, Codex.

Or install kafka-skills, the plugin that ships this one along with the rest of its 10 skills.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

README.md

[![agentmods](https://agentmods.dev/badge/skills/lensesio/agentic-engineering-for-apache-kafka/kafka-python-client.svg)](https://agentmods.dev/skills/lensesio/agentic-engineering-for-apache-kafka/kafka-python-client)

Your own site

<a href="https://agentmods.dev/skills/lensesio/agentic-engineering-for-apache-kafka/kafka-python-client"><img src="https://agentmods.dev/badge/skills/lensesio/agentic-engineering-for-apache-kafka/kafka-python-client.svg" alt="Measured on agentmods" height="20"></a>

Per session 150 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.

When invoked 5,377 The whole file, excluding the scripts and references it only reads on demand.

Security scan C 2 findings. Scan, not verified.

Origin original No closer match found in the catalogue.

Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

Model	Per session	Once invoked
Fable 5	$0.00150	$0.05377
Opus 5	$0.00075	$0.02688
Sonnet 5	$0.00030	$0.01075
Haiku 4.5	$0.00015	$0.00538

Measured 5d ago against content hash fe7cc40b802d, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade C, and why

kafka-python-client scanned grade C with 2 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 5d ago.

The scan reads SKILL.md. This mod also ships 5 executable files (evals/files/trips_api/main.py, references/common-template.py, references/consumer-template.py, …), listed below but not scanned — reading those needs a real analyzer, not pattern matching.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Downloads and executes remote codehighSupply chain

curl | sh runs whatever the server returns today, which is not necessarily what it returned when this was reviewed.

If `uv` is missing on the user's machine, point them at the official installer (`curl -LsSf https://astral.sh/uv/install.sh | sh`) before continuing — do not fall back to `pip`.

Makes network callslowCapability

Not a fault in itself. Listed so you know the mod talks to something, and to what.

If `uv` is missing on the user's machine, point them at the official installer (`curl -LsSf https://astral.sh/uv/install.sh | sh`) before continuing — do not fall back to `pip`.

skills/kafka-python-client/SKILL.md · 367 lines

How it starts

The opening of the file, as written. The whole thing — 367 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Kafka Python Client Scaffold

Generates a production-ready Python project that produces to and consumes from a Kafka topic using confluent-kafka-python, with Schema Registry (JSON Schema), graceful shutdown, idempotent producer, header-based schema identification, and tests. The agent should discover everything about the target topic (name, partitions, registered schema) from the live cluster via whichever Kafka MCP server is attached — Lenses MCP, Confluent's, Aiven's, or any other — before asking the user. Only fall back to questions if no Kafka MCP is attached or discovery returns nothing.

Target environment and keyword: $ARGUMENTS

Open your first reply with: "Running the kafka-python-client skill to scaffold this project."

Workflow

Copy this checklist and track your progress:

Scaffold Progress:
- [ ] Step 1: Discover topic + schema via the attached Kafka MCP
- [ ] Step 2: Hard gate - confirm with user before generating
- [ ] Step 3: Generate the project files
- [ ] Step 4: Run pytest against the generated tests
- [ ] Step 5: Run validation gate (kafka-topic-audit + kafka-perf-review)
- [ ] Step 6: Hand back with run instructions

Discover topic + schema via whichever Kafka MCP is attached (see references/mcp-discovery.md)
Hard gate: recap what was discovered, confirm with user before generating anything
Generate the project using the templates in references/
Run pytest against the generated tests/ and fix any failures (fix the code, not the tests)
Run validation gate: invoke kafka-topic-audit against the target topic and kafka-perf-review against the generated producer.py and consumer.py
Hand back with run instructions and the validation-gate findings inline

Step 1: Discover topic + schema via any attached Kafka MCP

Read references/mcp-discovery.md for the full probing procedure, vendor-specific tool-name hints and fallbacks. The high-level shape:

Identify the attached Kafka MCP server by looking at what's available in the session. Common ones: mcp__Lenses__* (Lenses MCP — reference implementation), mcp__Confluent__*, mcp__Aiven__*, custom servers tagged for Kafka.
Discover the environment / cluster using whichever tool the MCP exposes (Lenses: list_environments; Confluent: list_clusters; others vary).
Search for candidate topics by keyword from the user's prompt (Lenses: list_datasets(search=...); raw Kafka admin MCPs: list_topics then filter).
Fetch the registered schema for the chosen topic's value subject (Lenses: get_dataset; Confluent: get_schema(subject=<topic>-value); bare Schema Registry MCPs: HTTP GET against /subjects/<topic>-value/versions/latest).
Read partition count and replication factor (Lenses: get_topic_metadata; most others expose a describe_topic or get_topic equivalent).
Check existing consumer groups to suggest a non-colliding GROUP_ID (Lenses: list_consumer_groups_by_topic; most others: list_consumer_groups).

Read the full file on GitHub · 367 lines

Files

What ships with it

10 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

5d ago First seen · 367 lines · 150 tokens per session scan C fe7cc40b802d

Subscribe to this mod's changes

kafka-python-client is a skill published in the GitHub repository lensesio/agentic-engineering-for-apache-kafka (57 stars, last pushed 14d ago), licensed MIT. It adds 150 tokens to every session and 5,377 once invoked, about $0.0007 per session on Opus 5. A static security scan graded it C with 2 findings (downloads and executes remote code, makes network calls). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Backend and APIs Languages Python agent-skills agentic-engineering apache-kafka claude-code cursor data-engineering

python-data-engineering-and-pipeline-packaging

Guides agents through Python-based data engineering implementation. Use when building or modifying Python ingestion jobs, orchestration helpers, PySpark entry points, validation code, packaging, dependency management, or operational CLI workflows.

vaquarkhan/data-engineering-agent-skills · 51 tokens

temporal-python-testing

Test Temporal workflows with pytest, time-skipping, and mocking strategies. Covers unit testing, integration testing, replay testing, and local development setup. Use when implementing Temporal workflow tests or debugging test failures.

wshobson/agents · 45 tokens

fastapi-templates

Create production-ready FastAPI projects with async patterns, dependency injection, and comprehensive error handling. Use when building new FastAPI applications or setting up backend API projects.

wshobson/agents · 37 tokens

python-guidelines

This skill should be used when writing, reviewing, or refactoring Python code. Covers code integration, idiomatic patterns, docstring formatting, anti-abstraction rules, and software engineering basics.

fcakyon/claude-codex-settings · 42 tokens

manimgl-best-practices

Trigger when: (1) User mentions "manimgl" or "ManimGL" or "3b1b manim", (2) Code contains from manimlib import , (3) User runs manimgl CLI commands, (4) Working with InteractiveScene, self.frame, self.embed(), ShowCreation(), or ManimGL-specific patterns. Best practices for ManimGL (Grant Sanderson's 3Blue1Brown…

calesthio/OpenMontage · 167 tokens

authoring-dags

Workflow and best practices for writing Apache Airflow DAGs. Use when creating a new DAG, write pipeline code, handling questions about DAG patterns and conventions or extending an existing DAG with a follow-up/downstream task. ANY request shaped like 'add a DAG named X', 'write a pipeline', 'add a task that runs…

astronomer/agents · 93 tokens

kafka-python-client