hermes-model-auto-switch: Instructions file for Codex

Unattended Installation Runbook for hermes-model-auto-switch

What this repo is: This repository provides a hybrid auto-switch system for Hermes to ensure you always have at least one working model with a SWE score > 71%. It combines an FCM router daemon, NVIDIA NIM health monitoring, and a learning database.

Install

Run the following commands in order:

python install.py

Expected output: A series of OK / SKIP steps completing the setup, followed by a success summary.

Verify

Run these checks to verify a successful installation:

1. Verification Script:

python install.py --check

Pass condition: Script exits with 0 and prints no errors.

1. Daemon Health:

curl http://localhost:19280/health

Pass condition: returns JSON containing "ok":true and "running":true. (There is no status field on this endpoint.)

1. NIM Health Check:

python scripts/nvidia_nim_health_check.py

Pass condition: Outputs healthy NVIDIA models.

1. Hybrid Auto-Switch Dry Run:

python scripts/hybrid_auto_switch.py --dry-run

Pass condition: exits without a traceback and prints a JSON object.

Note the switcher is disabled by default. With NIM_AUTO_SWITCH_ENABLED unset it short-circuits and prints {"status": "disabled", ...} — that is expected, not a failure. To exercise the real path:

NIM_AUTO_SWITCH_ENABLED=true python scripts/hybrid_auto_switch.py --dry-run

Pass condition: prints JSON whose status is one of would_switch, no_change, or no_eligible_models, and config.yaml is byte-identical before and after. The JSON always carries fcm_resynced (always false under --dry-run — resync never runs in a rehearsal); retired_models and throttled_models appear at the top level when status is no_eligible_models, or nested as nim_health_summary.retired / nim_health_summary.throttled otherwise. A status of no_eligible_models means no monitored model is currently healthy — an unknown or sub-threshold SWE score no longer excludes a model by itself; the filter is healthy AND (SWE >= 71% OR SWE unknown) — see Known limitations in RELEASE_NOTES.md. Set NIM_AUTO_SWITCH_ENABLED=true permanently by adding it to the Hermes .env (hermes config env-path).