llm-interface-control

Local LLM Agent for Hardware Interface Control on Jetson

Use a local LLM running on Jetson to translate natural language commands into structured JSON for controlling hardware interfaces (lights, fans, thermostats, speakers) via GPIO, PWM, and I2C. The system uses Ollama with a custom prompt, FastAPI for the API layer, and confidence-based safety gating.

---

Execution model

Run one phase at a time. After each phase:

• Relay all output to the user.
• If output contains [STOP] → stop, consult the failure decision tree.
• If output ends with [OK] → tell the user "Phase N complete" and proceed.

---

Prerequisites

Requirement	Detail
Hardware	NVIDIA Jetson (Orin NX recommended) with JetPack
Python	3.8+
LLM Server	Ollama (installed in Phase 1)
Network	Local access for API calls

---

Phase 1 — Install Ollama (~2 min)

curl -fsSL https://ollama.com/install.sh | sh

Verify:

ollama --version

[OK] when Ollama version is printed. [STOP] if install fails — check network connectivity.

---

Phase 2 — Clone the project and install dependencies (~2 min)

git clone https://github.com/kouroshkarimi/llm_interface_controll.git
cd llm_interface_controll
pip install -r requirements.txt

[OK] when all packages install (FastAPI, uvicorn, etc.). [STOP] if pip fails — check Python version.

---

Phase 3 — Create the custom Ollama model (~2 min)

The project includes a system prompt file that constrains the LLM to output only structured JSON with fields: intent, device, action, location, parameters, confidence.

cd llm_interface_controll
ollama create jetson-controller -f models/jetson-controller.txt

Verify:

ollama list | grep jetson-controller

[OK] when jetson-controller appears in the model list. [STOP] if creation fails — ensure the base model (llama3.2:1b) is available: ollama pull llama3.2:1b.

---

Phase 4 — Start the FastAPI server (~1 min)