spark-history-cli: Instructions file for Codex

AGENTS.md

Project Overview

spark-history-cli is a Python CLI for querying the Apache Spark History Server REST API. It also ships two agent skills (spark-history-cli and spark-advisor) that teach AI coding assistants how to debug and optimize Spark applications.

Repository Structure

spark-history-cli/
├── spark_history_cli/       # Python package
│   ├── cli.py               # Click-based CLI entry point (REPL + one-shot)
│   ├── core/                # API client, formatters, data models
│   ├── utils/               # Skill installer, helpers
│   ├── tests/               # pytest test suite
│   └── skills/              # Bundled SKILL.md (installed via `install-skill` command)
├── skills/                  # Root-level skills (discovered by `npx skills add`)
│   ├── spark-history-cli/   # CLI usage skill
│   │   ├── SKILL.md
│   │   └── sample_codes/
│   └── spark-advisor/       # Performance diagnosis skill
│       ├── SKILL.md
│       ├── references/      # diagnostics.md, comparison.md
│       └── sample_codes/
├── setup.py                 # Package metadata, version, entry points
├── CHANGELOG.md             # Release history
└── .github/workflows/
    ├── ci.yml               # Tests on push/PR
    └── publish.yml          # PyPI publish on GitHub Release

Key Development Guidelines

• Python 3.10+ required
• Install locally: pip install -e .
• Run tests: pytest spark_history_cli/tests/
• Lint: standard Python conventions, no custom linter configured
• Entry point: spark_history_cli.cli:main

Making Changes

CLI commands

All commands are in cli.py using Click. Each command calls methods on the API client in core/. Add new commands by following the existing pattern (Click command → API call → format output).

Skills

Skills are pure Markdown (SKILL.md) with optional references/ and sample_codes/. The skills in skills/ (root) are the canonical source — spark_history_cli/skills/ is a bundled copy for the install-skill command.