kai-deploy

Deploy an export on KAI

You are handed an export — observations/<key>/exports/<token>.yaml, or just its run token (rt65y1). Your job is to serve its placement on the kai-gpu node pool and leave it either placed exactly as its slots say, or torn down with the reason stated.

This is a separate step and the user starts it. Nothing in the pipeline deploys a result, and run-fleet never chains into this. The guard hook asks before any colocation.kai deploy, load or teardown unless the user's own prompt was /fleet-kai-deploy, /fleet-kai-load or /fleet-kai-teardown; an ask is the rule working, not something to route around.

Everything runs from the repository's tutorials/multi-model-gpu-sharing directory with PYTHONPATH=src.

A split's export is one deployment per node. An export with schema_version: 2 (written by search on a fleet with max_nodes above 1, <store>/groups/<fleet>/export.<stamp>.yaml) holds each node's own group export. render and deploy take it like any export, and each node becomes a deployment of its own, <name>-n1, <name>-n2, ..., deployed one after the other exactly as a single node is. The first tenant of each later node is kept off the earlier nodes' hosts and in the first node's zone. It therefore needs that many kai-gpu nodes of the instance type in one zone, and capacity for that is not guaranteed, so say so before deploying. Both render and deploy refuse (exit 2) if a node's own export file no longer matches the copy in the split, because that means the group was exported again since. Every per-deployment step below (retry, verify, drift, teardown) takes a node's <name>-n<i>, and a mismatch on one node is retried by tearing down every node of the split and deploying again, since later nodes were placed relative to earlier ones.

What the code decides, so you do not

• Sizing. render re-derives each tenant's envelope offline, exactly as export did, and refuses (exit 2) if any differs from the export's required_gib or if the fleet file's sha256 moved. Each shard becomes a fraction L = (E - r_mps) + r_kai with r_kai = r_mps + 0.125 GiB, and util is computed against L, which hands vLLM the budget it was measured with.
• Filling. Every slot's card is filled to the store's addressable_mib less 256 MiB, the extra going to the slot's single-shard tenants as KV. That is what makes KAI's binpack reproduce the slots; it is not a choice to revisit per deploy.
• Order. Cards without a TP tenant first, then TP tenants, then their mates — each tenant waited on until the binder has written its cards, and every tenant after the first pinned to the first one's node. Each tenant is a Helm release of the kai-vllm chart, kai-<name>-<tenant>, installed without --wait (bound is what the next one needs, not Ready), and the plan is a release of its own, kai-<name>-plan. Helm orders nothing between releases; deploy does.
• SM caps. A tenant the search measured under an SM cap carries active_thread_percentage in the export, and is served with CUDA_MPS_ACTIVE_THREAD_PERCENTAGE set to it in its pod env, which KAI's shared MPS server honours. render prints SM cap N% beside it. An uncapped tenant gets no variable.