strands-sglang AGENTS.md

AGENTS.md

Guidance for coding agents working in this repository. CLAUDE.md is a symlink to this file.

Project Overview

Strands-SGLang is an SGLang model provider for the Strands Agents SDK with Token-In/Token-Out (TITO) support for on-policy agentic reinforcement learning training. It captures exact token IDs and logprobs during generation without retokenization drift, which is critical for accurate gradient computation in RL training.

Commands

Setup

uv sync              # installs the dev group by default
pre-commit install   # hook types come from default_install_hook_types

Linting

pre-commit run --all-files   # what CI's lint job runs; the tools below are a subset
ruff check src/ tests/ examples/
ruff format --check src/ tests/ examples/
mypy src/strands_sglang

Testing

# Unit tests (no server needed)
pytest tests/unit/ -v

# Single test file
pytest tests/unit/test_sglang.py -v

# Single test
pytest tests/unit/test_tool_parser.py::TestHermesToolParser::test_parse_single_tool_call -v

# Unit tests with coverage
pytest tests/unit/ -v --cov=src/strands_sglang --cov-report=html

# Integration tests (requires running SGLang server)
pytest tests/integration/ -v --sglang-base-url=http://localhost:30000
# Or via env var: SGLANG_BASE_URL=http://localhost:30000 pytest tests/integration/

Architecture

The package lives in src/strands_sglang/ with 7 core modules:

SGLangModel (sglang.py) - Main entry point implementing the Strands Model interface. Requires client and tokenizer (keyword-only). Formats messages using HuggingFace chat templates (apply_chat_template()), calls SGLang's /generate endpoint (non-streaming by design for RL throughput), tracks TITO trajectory, and parses tool calls. VLM support is auto-detected server-side via SGLangClient.is_multimodal() (queries /model_info for has_image_understanding, cached after the first call). When multimodal, collect_image_data() derives image_data (base64 data URLs) from the messages on every call and forwards them to SGLang — the server handles image token expansion. Configuration via SGLangConfig TypedDict (sampling_params, return_logprob, return_routed_experts, enable_thinking), which inherits context_window_limit from Strands' BaseModelConfig — set it, or conversation managers and estimate_utilization() fall back to a hardcoded 200k. reset() starts a new trajectory, banking the finished one in rollout_history.