Getting it into your agent
It runs from inside its repository, so the clone comes first — what it calls does not travel with the file alone.
git clone --depth 1 https://github.com/Seeed-Projects/Seeed-Jetson-DevelopToolnpx agentmods add skills/seeed-projects/seeed-jetson-developtool/llama-cpp-rpc-distributedWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/seeed-projects/seeed-jetson-developtool/llama-cpp-rpc-distributed)<a href="https://agentmods.dev/skills/seeed-projects/seeed-jetson-developtool/llama-cpp-rpc-distributed"><img src="https://agentmods.dev/badge/skills/seeed-projects/seeed-jetson-developtool/llama-cpp-rpc-distributed/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/seeed-projects/seeed-jetson-developtool/llama-cpp-rpc-distributed"><img src="https://agentmods.dev/badge/skills/seeed-projects/seeed-jetson-developtool/llama-cpp-rpc-distributed.svg" alt="Reviewed on agentmods" width="80" height="20"></a>- NVIDIA SkillSpector warn
SkillSpector: 4 findings, up to medium
These are SkillSpector’s own severities. On a checked sample its high-severity flags on skills were ~96% false positives — a documented command, a public API, a “never do X” rule — so we show them as a caution to read, not a verdict. Why →
- medium Privilege Escalation · line 40 Commands invoke sudo or root privileges. Verify this elevated access is necessary and justified.Fix: Avoid sudo/root unless strictly required. Prefer least-privilege patterns. If elevation is needed, document the justification and scope.
- medium Privilege Escalation · line 41 Commands invoke sudo or root privileges. Verify this elevated access is necessary and justified.Fix: Avoid sudo/root unless strictly required. Prefer least-privilege patterns. If elevation is needed, document the justification and scope.
- medium Privilege Escalation · line 177 Commands invoke sudo or root privileges. Verify this elevated access is necessary and justified.Fix: Avoid sudo/root unless strictly required. Prefer least-privilege patterns. If elevation is needed, document the justification and scope.
- medium Privilege Escalation · line 178 Commands invoke sudo or root privileges. Verify this elevated access is necessary and justified.Fix: Avoid sudo/root unless strictly required. Prefer least-privilege patterns. If elevation is needed, document the justification and scope.
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00062 | $0.01345 |
| Opus 5.5 | $0.00025 | $0.00538 |
| Sonnet 5.5 | $0.00012 | $0.00269 |
| Haiku 4.5 | $0.00006 | $0.00135 |
Grade B, and why
llama-cpp-rpc-distributed scanned grade B with 1 finding against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 1mo ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Asks for rootmediumPrivilege escalation
A mod that escalates privileges can change anything on the machine, not only the project.
sudo apt update How it starts
The opening of the file, as written. The whole thing — 189 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Distributed llama.cpp on Jetson (RPC Mode)
Run large language models across multiple Jetson devices by leveraging llama.cpp's RPC backend. This enables horizontal scaling — split model layers across GPUs on different machines connected via LAN.
Execution model
Run one phase at a time. After each phase:
- Relay all output to the user.
- If output contains
[STOP]→ stop, consult the failure decision tree. - If output ends with
[OK]→ tell the user "Phase N complete" and proceed.
Prerequisites
| Requirement | Detail |
|---|---|
| Hardware | Two reComputer Jetson devices (e.g. Orin NX/AGX Orin) |
| JetPack | 6.x+ with working CUDA drivers |
| Network | Both devices on same LAN, able to ping each other |
| RAM | Client ≥ 64 GB, remote node ≥ 32 GB (unified memory) |
| Storage | ~5 GB free for build + model |
Phase 1 — Clone and install build dependencies (~2 min)
Run on both machines:
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
sudo apt update
sudo apt install -y build-essential cmake git libcurl4-openssl-dev python3-pip
[OK] when clone and apt install complete on both machines.
Phase 2 — Build with RPC + CUDA backend (~5 min)
Run on both machines:
cd llama.cpp
cmake -B build \
-DGGML_CUDA=ON \
-DGGML_RPC=ON \
-DCMAKE_BUILD_TYPE=Release
cmake --build build --parallel
Verify:
ls build/bin/llama-cli build/bin/rpc-server
[OK] when both llama-cli and rpc-server binaries exist.
[STOP] if cmake or build fails — see failure decision tree.
Phase 3 — Install Python conversion tools (~1 min)
Run on the client machine (Machine A):
cd llama.cpp
pip3 install -e .
[OK] when pip install completes.
Phase 4 — Download and convert model (~5–10 min)
Using TinyLlama-1.1B-Chat as an example:
pip3 install huggingface-hub
huggingface-cli download TinyLlama/TinyLlama-1.1B-Chat-v1.0 --local-dir ~/TinyLlama-1.1B-Chat-v1.0
What ships with it
2 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 1mo ago First seen · 189 lines · 62 tokens per session scan B 8d4cd515fb9b
llama-cpp-rpc-distributed is a skill published in the GitHub repository Seeed-Projects/Seeed-Jetson-DevelopTool (56 stars, last pushed yesterday), licensed MIT. It adds 62 tokens to every session and 1,345 once invoked, about $0.0002 per session on Opus 5.5. A static security scan graded it B with 1 finding (asks for root). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
spark-environment-setup
Set up a working ML training/inference environment on NVIDIA DGX Spark (GB10, aarch64, CUDA 13). Use when installing PyTorch/Unsloth/TRL/vLLM on DGX Spark, hitting libcudart or wheel-ABI errors on aarch64, or choosing between NGC containers and bare pip installs.
spark-memory-thermal-ops
Manage unified memory and thermals during long-running ML jobs on NVIDIA DGX Spark. Use when planning memory headroom for a training run on GB10, when a job OOMs on unified memory, or when monitoring temperature and power during multi-hour training.
spark-training-gotchas
Preflight and diagnose the ten known failure modes for ML training on NVIDIA DGX Spark. Use when a training run on DGX Spark fails to start, OOMs below the 128GB limit, slows down mid-run, or before any multi-hour training job on GB10.
inference-serving-llama-cpp
Runs LLM inference on CPU, Apple Silicon, and consumer GPUs without NVIDIA hardware. Use for edge deployment, M1/M2/M3 Macs, AMD/Intel GPUs, or when CUDA is unavailable. Supports GGUF quantization (1.5-8 bit) for reduced memory and 4-10× speedup vs PyTorch on CPU.
amc-run-rtsp-calibration
Calibrate a new dataset from live RTSP camera streams via the AutoMagicCalib REST API. Use when the user provides RTSP URLs or asks to calibrate live cameras; VIOS records clips, AMC ingests them, then runs calibration.
deepstream-generate-pipeline
Build DeepStream GStreamer pipelines interactively. Use when the user asks about pipelines for video/image inference, detection, tracking, or streaming — including natural phrases like 'pipeline to infer on image', 'run inference on video', 'detect objects in stream', 'save inference output', 'deepstream pipeline'…