local-llm-text-to-image

Text-to-Image with Stable Diffusion on Jetson

Generate images from text prompts locally on Jetson using Stable Diffusion. Three deployment paths: KerasCV with TensorFlow, Hugging Face diffusers with PyTorch, and NVIDIA's containerized AUTOMATIC1111 WebUI.

---

Execution model

Run one phase at a time. After each phase:

• Relay all output to the user.
• If output contains [STOP] → stop, consult the failure decision tree.
• If output ends with [OK] → tell the user "Phase N complete" and proceed.

---

Prerequisites

Requirement	Detail
Hardware	Jetson Orin NX 16GB (e.g. reComputer J4012)
JetPack	5.1.1+
Storage	~20 GB free (models are large)
RAM tips	Disable desktop GUI to save ~800 MB. Consider disabling ZRAM and using swap.

---

Path A — KerasCV Stable Diffusion

Phase A1 — Create virtual environment and install TensorFlow (~10 min)

sudo apt install python3.8-venv
python3 -m venv kerasStableEnvironment
source kerasStableEnvironment/bin/activate
cd kerasStableEnvironment
pip install -U pip
pip install -U numpy grpcio absl-py py-cpuinfo psutil portpicker six mock requests gast h5py astor termcolor protobuf keras-applications keras-preprocessing wrapt google-pasta setuptools testresources

Install TensorFlow for your JetPack version (example for JP 5.1.1):

pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v511 tensorflow==2.12.0+nv23.05

Verify:

python -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"

[OK] when GPU device is listed.

Phase A2 — Install PyTorch and KerasCV (~5 min)

sudo apt install libopenblas-dev
pip install --no-cache https://developer.download.nvidia.com/compute/redist/jp/v511/pytorch/torch-2.0.0+nv23.05-cp38-cp38-linux_aarch64.whl
pip install keras-cv==0.5.1 keras==2.12.0 Pillow

Verify PyTorch CUDA:

python -c "import torch; print(torch.cuda.is_available())"