switchyard-testing-ci

Switchyard Testing And CI

Read .github/workflows/ci.yml before claiming local CI equivalence. Select tests from the current diff and symbol references rather than a static subsystem inventory.

Validation Matrix

Change	Minimum trustworthy validation
Markdown or skill Markdown only	Parse skill frontmatter when applicable; git diff --check
Python implementation	Focused pytest, uv run ruff check ., uv run mypy switchyard
Broad Python or public API	Hermetic full pytest plus lint and mypy
One Rust crate	cargo fmt --all --check, workspace clippy, focused crate tests
Shared Rust/API/FFI	Rust workspace tests plus affected Python tests
Packaging, extras, top-level imports	The slim-install smoke commands from the current CI workflow
Live provider behavior	Focused integration test only after explicit authorization

Standard Gates

Python pre-PR gate:

uv run ruff check .
uv run mypy switchyard
env -u OPENROUTER_API_KEY -u NVIDIA_API_KEY -u OPENAI_API_KEY \
  -u ANTHROPIC_API_KEY uv run pytest tests/ -v -m "not integration"

Rust workspace gate:

cargo fmt --all --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace

Use -o addopts= when the repository's pytest defaults would stop failure collection too early. Run focused tests first, then broaden only when the changed contract or shared ownership warrants it.

Live Tests

Integration tests can spend money, require credentials, and contact real services. Never run them as an automatic pre-PR step. Provision NVIDIA_API_KEY through the environment or a secret manager before running the user-authorized path, and state which provider was called:

uv run pytest tests/e2e/<focused_test>.py \
  -v -m integration -o addopts=

Skill Validation

Each retained skill must have YAML frontmatter with name and description:

uv run python - <<'PY'
from pathlib import Path

import yaml

for path in Path(".agents/skills").glob("*/SKILL.md"):
    parts = path.read_text().split("---", 2)
    data = yaml.safe_load(parts[1]) if len(parts) == 3 else None
    if not isinstance(data, dict) or not data.get("name") or not data.get("description"):
        raise SystemExit(f"{path}: missing name or description")
print("skill frontmatter is valid")
PY
git diff --check