training_hub: Instructions file for Codex

AGENTS.md - Training Hub

Guidelines for AI agents working in this codebase.

Project Overview

Training Hub is an algorithm-focused interface for common LLM training, continual learning, and reinforcement learning techniques. The goal is to expose common training algorithms in an intuitive and easy-to-use way, abstracting away backend complexity. Training Hub is designed to support as many backends as necessary—the current implementations are just the starting point.

• Language: Python 3.11+
• License: Apache-2.0
• Primary Author: Red Hat AI Innovation Team

For the current list of supported algorithms, backends, and dependencies, see:

• pyproject.toml - Dependencies and optional extras
• src/training_hub/__init__.py - Public API exports
• README.md - User-facing documentation and support matrix

Quick Commands

# Install in editable mode (development)
pip install -e .

# Install with LoRA support
pip install -e .[lora]

# Install with GRPO support (includes ART + verl backends)
pip install -e .[grpo,lora]

# Install with CUDA support (install sequentially after other extras)
pip install -e .[cuda] --no-build-isolation

# Install with development dependencies
pip install -e .[dev]

# Run tests
pytest tests/

# Serve documentation locally (requires docsify-cli)
cd docs && docsify serve

See pyproject.toml for the full list of optional dependency groups.

Code Organization

src/training_hub/
├── __init__.py              # Public API exports
├── hub_core.py              # Core utilities
├── utils.py                 # Shared utilities (torchrun params, type formatting)
├── visualization.py         # plot_loss() for training curves
├── algorithms/
│   ├── __init__.py          # Base classes: Algorithm, Backend, AlgorithmRegistry
│   ├── sft.py               # Supervised Fine-Tuning
│   ├── osft.py              # Orthogonal Subspace Fine-Tuning
│   ├── lora.py              # LoRA + SFT
│   ├── lora_grpo.py         # LoRA + GRPO and GRPO (ART backend, algorithm, convenience fns)
│   ├── lora_grpo_verl.py    # verl backend for LoRA + GRPO and GRPO
│   ├── rewards.py           # Reward functions (tool_call_reward, binary_reward)
│   ├── verl_tool_agent.py   # Custom verl agent loop for tool-call training
│   └── peft_extender.py     # PEFT parameter handling for LoRA
└── profiling/
    └── memory_estimator.py  # GPU memory estimation for training