local-ai-query

Skill: local-ai-query

Purpose

Route prompts to a locally running Ollama instance instead of cloud AI APIs. Use this for privacy-sensitive tasks, offline work, or cost reduction. Pairs with the Docker stack in docker/local-ai-stack/.

When to Use

• Processing sensitive/proprietary code or data that shouldn't leave the machine
• Offline or air-gapped environments
• Rapid iteration where cloud latency matters
• Cost-conscious experimentation with large prompts

Prerequisites

• Ollama running locally (see docker/local-ai-stack/README.md)
• At least one model pulled: docker exec ollama ollama pull llama3.2:latest

Steps

1. Verify Ollama is available

curl -s http://localhost:11434/api/tags | jq '.models[].name'

If this fails, start the stack: cd docker/local-ai-stack && docker compose up -d

2. Choose appropriate model

Task	Recommended Model
Code generation	deepseek-coder-v2:latest or codellama:latest
General chat	llama3.2:latest
Summarization	llama3.2:latest
Embeddings/RAG	nomic-embed-text:latest
Fast/small	phi3:mini

Pull a model if needed:

docker exec ollama ollama pull <model-name>

3. Send query to local model

Simple generation:

curl http://localhost:11434/api/generate \
  -d '{
    "model": "llama3.2:latest",
    "prompt": "<YOUR_PROMPT>",
    "stream": false
  }' | jq -r .response

Chat format (multi-turn):

curl http://localhost:11434/api/chat \
  -d '{
    "model": "llama3.2:latest",
    "messages": [
      {"role": "user", "content": "<YOUR_MESSAGE>"}
    ],
    "stream": false
  }' | jq -r '.message.content'

With system prompt:

curl http://localhost:11434/api/chat \
  -d '{
    "model": "llama3.2:latest",
    "messages": [
      {"role": "system", "content": "You are a senior software engineer. Be concise and technical."},
      {"role": "user", "content": "<YOUR_MESSAGE>"}
    ],
    "stream": false
  }' | jq -r '.message.content'