delegating-to-local-llm

Delegating to a Local LLM

The subagent is an interactive Pi session pinned to a local omlx model, spawned in a herdr pane. herdr provides the whole control loop — spawn, state, read, send — and Pi's herdr integration reports working/blocked/idle automatically. The session stays visible and steerable; you own acceptance of its work.

Preflight

uname -m                                                  # arm64 required (omlx is Apple-Silicon MLX)
KEY=$(jq -r .auth.api_key ~/.omlx/settings.json)
curl -s --max-time 3 -H "Authorization: Bearer $KEY" http://127.0.0.1:8000/v1/models
which pi && herdr agent list >/dev/null && echo ok

• curl fails -> omlx start; succeeds but lists no models -> models need downloading
• pi or herdr missing, not arm64, or first run on this machine -> walk the user through references/setup.md

Delegate

1. Pick the model from the /v1/models response. The user's named choice wins; otherwise ask which model they prefer for the task. Memory headroom is model- and machine-specific: if omlx's prefill guard aborts mid-task ("Prefill context too large for available memory"), the model cannot handle the accumulated context on this machine — restart the task on a lighter model.

2. Spawn — short kebab task name; cwd is the project the task touches. Pin the pane to your own workspace — spawning defaults to whichever workspace the user has focused at that moment, which may not be yours:

   WS=$(herdr pane get "$HERDR_PANE_ID" | jq -r '.result.pane.workspace_id')
   herdr agent start <task-name> --cwd <dir> --workspace $WS --no-focus -- omlx launch pi --model <model-id> --api-key $KEY
   

   Capture pane_id from the JSON response. (omlx launch also rewrites Pi's omlx provider config, so it is always current.)

3. Wait ready: herdr agent wait <task-name> --status idle --timeout 30000

4. Send the task, then submit it:

   herdr agent send <task-name> "<task>"
   sleep 1 && herdr pane send-keys <pane-id> enter