devtu-remote-provider

ToolUniverse remote provider

A remote tool is a model you host yourself, exposed over MCP. The contract lives in src/tooluniverse/remote_resident/STANDARD.md; read it before changing a provider. This page is how to apply it and how to read what it refuses.

The shape

agent ──MCP──> remote/<slug>/<slug>_mcp_server.py     ToolUniverse's env, CPU only
                      │
                      │ Unix socket, one request at a time
                      ▼
               remote/<slug>/resident/<slug>_worker.py   the model's own env

Two environments because their pinned stacks conflict. Boltz-2 needs torch 2.12.0+cu130 / triton 3.7.0 / numpy 1.26.4, which cannot be installed alongside ToolUniverse's other 1000+ tools. The MCP server must not import the model's dependencies, and the worker must not import ToolUniverse — the shared surface is one dependency-free file (remote_resident/protocol.py) loaded by path.

Top mistakes

1. Requiring the model's CLI at tool construction. shutil.which("boltz") in __init__ fails on a correctly separated deployment. Accept either route.
2. Trusting weights because the download exited 0. A boltz2_conf.ckpt arrived as 1.2 % of itself and served predictions for a month. Declare size and sha256; nothing is fetched implicitly at request time.
3. Starting the model per request. Measured on a GB10: 110 s per call as a subprocess, ~54 s held. Most of the difference is not kernels — it is CUDA graphs captured on one request and replayed on the next, which a worker that exits cannot collect.
4. Letting a worker outlive its server. start_new_session=True keeps a Ctrl-C from killing a prediction, and also orphans the worker; SIGKILLing the MCP server left two processes holding the GPU. Workers need the idle timeout and the orphan watchdog.
5. Claiming status = "verified" without hardware. It must carry verified_on. A bare "verified" claims every machine.
6. Hashing gigabytes on every check. Use the digest memo; pass deep=True only in setup, where what is certified should have been read.
7. Forgetting package data. A built wheel carries only .py under remote/. Provider JSON, provider.toml, lock files, smoke inputs and the R halves of R-backed providers need globs in pyproject.toml.
8. Believing exit 0. OpenFold3's smoke exited 0 in 29 s, reported a GPU peak, and produced no structure: upstream caught the per-query exception, wrote "Failed Queries: 1" to a summary file and returned normally. Declare [runtime] outputs (globs) or output_keys (reply fields); the manifest refuses status = "verified" without one and the smoke step refuses when nothing matched.
9. Quoting a budget from the smoke. RFdiffusion3's 60-mer smoke reserves 2.12 GiB; a 400-residue, 4-design request reserves 12.49. When peak tracks input size, measure a ladder and declare the envelope alongside the number.
10. Forking a loader out of a resident worker. The parent has CUDA and the kit's threads initialised long before a request arrives. Set num_workers = 0 unconditionally — it is a property of the resident route, not of one kit.
11. Accepting a smoke as verification. One request is not a test of a resident provider. openfold3 produced five real structures and then failed deterministically on the SECOND request of the same shape, because a lever dropped a CUDA graph on moved storage while keeping its emptied pool handle. Run at least four consecutive identical requests, then a mixed-shape sequence. Four different shapes pass on a build that cannot serve a repeat.
12. Assuming levers_off reaches the kit. boltzgen's modes.py never reads MODEL_OPT_LEVERS_OFF, and its stack.activate() overwrites a caller's value for any switch the mode exports. Check the ACTIVE line: if the lever is still listed the door did not open, and shipping the setting anyway is worse than saying so.
13. Quoting wall times from a contended card. The same fold read 15-19 s quiet and 28-56 s with another session on the GPU. Peak reserved reproduced to the hundredth of a GiB in both. Report peaks from any run; report wall times only from an idle card. And utilisation alone is not evidence that YOUR process is computing — a 40-minute boltzgen stall was read as progress because the card showed 86-91%, which belonged to someone else. The usable signal is utilisation plus /proc/<pid>/stat CPU, on a quiet card.
14. Letting a library fork. num_workers = 0 covers the loader and nothing else. boltzgen's DataLoader worker sat in torch.cdist → at::parallel_for → an OpenMP region, alive and at 0% CPU, waiting on a pool fork() did not carry over — up only because the kit's levers had done host-tensor work during activation. The rule that covers all three observed shapes: a resident worker that has run any host-tensor work must not let a library fork, and [kit.env] OMP_NUM_THREADS = "1" is the cheapest guarantee.
15. An outputs glob that matches a file written regardless of success. boltzgen's configure writes design_spec.cif — a picture of the specification — whether or not anything is designed, so *.cif would have passed a run that produced nothing. Test the glob against a failing run, not only a passing one.
16. Reading four identical requests as the acceptance test. They are the WEAKER half. A held engine is built once and replayed, so N identical requests exercise exactly one build — they cannot show the worker can build a SECOND one. genie3 would have passed four and raised on the first request that changed its engine key, because upstream's sampling config is a module-level mutable permitting one build per process. Run four identical requests then at least one that changes the engine key; the rebuild is the danger, not the new shape. Before writing a worker, call the upstream config or model builder twice in one process — one minute, and invisible to the repeats test.
17. An input ceiling that the budget cannot cover. progen2's tool allowed 16 sequences × 1024 tokens; each is sane, the product costs 116.93 GiB, essentially the whole card. Bound the product, and check the ceilings against the budget rather than only against each other.
18. Trusting a probe that does nothing afterwards. protenix's stock LayerNorm emits SASS with no PTX above sm_100, so the call returns, synchronize() PASSES, and the next unrelated op raises no kernel image is available. The first probe printed "LAUNCH OK" twice because it stopped there.
19. Assuming a wheel that installs will run. tokenizers 0.10.x publishes well-formed aarch64 wheels that Illegal instruction (core dumped) at import. In a worker that is a smoke_protocolerror with no traceback, because SIGILL is uncatchable. Import every compiled pin in [env] verify.
20. Assuming pip honours requirement order. A lock with two CUDA generations gets whichever of a colliding pair pip reaches last — measured, libcudnn.so.9 at 124,920 bytes instead of 6,328,728. Declare ordered passes with # --- pass break ---.
21. Sizing an ablation to the smoke. Every kernel lever of the atlasfold kit is gated at min_tokens = 512; its 35-residue smoke exercises none. Seven ablated levers fold 35 residues and abort at 600; thirteen fold both. Size the ablation for the largest input the provider accepts — and check whether stock itself survives that size, because atlasfold's does not.