voice-llm-motor-control

Voice-Controlled Motor System on Jetson

Build a voice-to-motor pipeline on reComputer Robotics J4012: Whisper captures speech → Qwen 2.5 LLM interprets intent → CAN bus drives MyActuator X Series Motors. Users can say commands like "Rotate 90 degrees clockwise."

---

Execution model

Run one phase at a time. After each phase:

• Relay all output to the user.
• If output contains [STOP] → stop immediately, consult the failure decision tree.
• If output ends with [OK] → tell the user "Phase N complete" and proceed to the next phase.

---

Prerequisites

Requirement	Detail
Jetson device	reComputer Robotics J4012
Microphone	reSpeaker XVF3800
Motor	MyActuator X Series Motors (CAN bus)
JetPack	With CUDA 12.6
Network	Internet access for cloning repos and pulling models

---

Phase 1 — Install Whisper ASR server (~10 min)

Clone and build the Whisper server:

git clone https://github.com/jjjadand/whisper-stable4curl
cd whisper-stable4curl
export PATH=/usr/local/cuda-12.6/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda-12.6/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
cmake --build build -j --config Release

Launch the Whisper inference service (in this terminal):

./build/bin/whisper-stream -m ./models/ggml-base.en-q5_1.bin -t 8 --step 0 --length 7000 -vth 0.7 --keep 1200

[OK] when Whisper starts listening for audio input. [STOP] if cmake build fails or CUDA paths are wrong.

---

Phase 2 — Install Ollama and pull Qwen 2.5 (~10 min)

Open a new terminal and install Ollama:

curl -fsSL https://ollama.com/install.sh | sh

Pull the Qwen 2.5 model:

ollama pull qwen2.5

[OK] when ollama pull completes successfully. [STOP] if Ollama install fails or model download errors.

---

Phase 3 — Install and run motor control script (~5 min)

Clone the voice control project:

git clone https://github.com/yuyoujiang/voice_control.git
cd voice_control