A guide to llama.cpp, a C and C++ framework for running large language models on your own hardware. It explains model files, compression, hardware backends, an HTTP server, embeddings, and other supported interfaces.

Use it to build llama.cpp with CUDA, Vulkan, Metal, or ROCm, work with GGUF models, create an OpenAI-compatible server, generate embeddings, or add constrained and function-based model output.

It helps you understand the available building blocks and avoid guessing when compiling, configuring, or connecting llama.cpp to another application.

llama-cpp — Skill for Claude Code and Codex by maystudios

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

Any agent

npx skills add maystudios/claude-skills --skill llama-cpp

Clone the repo

git clone --depth 1 https://github.com/maystudios/claude-skills

Made for: Claude Code, Codex.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

README.md

[![agentmods](https://agentmods.dev/badge/skills/maystudios/claude-skills/llama-cpp/github.svg)](https://agentmods.dev/skills/maystudios/claude-skills/llama-cpp)

Your own site

<a href="https://agentmods.dev/skills/maystudios/claude-skills/llama-cpp"><img src="https://agentmods.dev/badge/skills/maystudios/claude-skills/llama-cpp/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

Your own site · 80×15

<a href="https://agentmods.dev/skills/maystudios/claude-skills/llama-cpp"><img src="https://agentmods.dev/badge/skills/maystudios/claude-skills/llama-cpp.svg" alt="Reviewed on agentmods" width="80" height="20"></a>

Per session 185 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.

When invoked 2,199 The whole file, excluding the scripts and references it only reads on demand.

Security scan A 1 finding. A grade says what 26 rules found in the file — not that it is safe. Third-party audits

NVIDIA SkillSpector warn 7 Sept 2026

SkillSpector: 1 finding, up to medium

These are SkillSpector’s own severities. On a checked sample its high-severity flags on skills were ~96% false positives — a documented command, a public API, a “never do X” rule — so we show them as a caution to read, not a verdict. Why →

medium Data Exfiltration · line 55
Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Fix: Verify the destination URL is trusted and necessary. Remove or replace with documented APIs. Ensure no secrets, tokens, or PII are transmitted.

How audits are shown

Origin original No closer match found in the catalogue.

Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

Model	Per session	Once invoked
Fable 5.1	$0.00185	$0.02199
Opus 5.5	$0.00074	$0.00880
Sonnet 5.5	$0.00037	$0.00440
Haiku 4.5	$0.00018	$0.00220

Measured 2mo ago against content hash 3d0cf6e2a8f9, method: parsed. Prices are Anthropic first-party input rates as of 2026-10-07, from the pricing page.

Security

Grade A, and why

llama-cpp scanned grade A with 1 finding against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2mo ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Makes network callslowCapability

Not a fault in itself. Listed so you know the mod talks to something, and to what.

# Query via curl

llama-cpp/SKILL.md · 187 lines

How it starts

The opening of the file, as written. The whole thing — 187 lines — stays where its author put it; the contents beside it link to each section on GitHub.

llama.cpp -- C/C++ LLM Inference Framework Guide

Official Documentation

Source	URL
GitHub Repository	https://github.com/ggml-org/llama.cpp
C API Header (llama.h)	https://github.com/ggml-org/llama.cpp/blob/master/include/llama.h
C++ RAII Wrappers	https://github.com/ggml-org/llama.cpp/blob/master/include/llama-cpp.h
Build Instructions	https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md
Server Documentation	https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md
Quantization Tool	https://github.com/ggml-org/llama.cpp/blob/master/tools/quantize/README.md
GGUF Specification	https://github.com/ggml-org/ggml/blob/master/docs/gguf.md
Function Calling Docs	https://github.com/ggml-org/llama.cpp/blob/master/docs/function-calling.md
Multimodal Docs	https://github.com/ggml-org/llama.cpp/blob/master/docs/multimodal.md
Examples Directory	https://github.com/ggml-org/llama.cpp/tree/master/examples
HuggingFace GGUF Hub	https://huggingface.co/docs/hub/gguf-llamacpp
Llama-Unreal Plugin	https://github.com/getnamo/Llama-Unreal

What is llama.cpp?

llama.cpp is a pure C/C++ LLM inference engine with minimal dependencies, designed for high-performance local inference across CPUs and GPUs. Key properties:

MIT licensed, extremely active development (~daily releases, currently b8766+)
Widest hardware support: NVIDIA (CUDA), AMD (ROCm/Vulkan), Apple (Metal), Intel (SYCL/Vulkan), Qualcomm (OpenCL), ARM, WebGPU
GGUF model format: single-file, mmap-compatible, 40+ quantization types from 1.5-bit to 16-bit
Built-in HTTP server: OpenAI-compatible API, Anthropic Messages API, streaming, function calling, multimodal
Language bindings: Python (llama-cpp-python), Go, Rust, C#, Node.js, Java, Swift, and more

Quick Start

Run a model via server (fastest path)

# Install
brew install llama.cpp    # macOS/Linux
winget install llama.cpp  # Windows

# Start server with a HuggingFace model
llama-server -hf bartowski/Llama-3.3-70B-Instruct-GGUF:Q4_K_M -ngl 99

# Query via curl
curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"Hello"}],"temperature":0.8}'

Read the full file on GitHub · 187 lines

Files

What ships with it

6 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

2mo ago First seen · 187 lines · 185 tokens per session scan A 3d0cf6e2a8f9

Subscribe to this mod's changes

llama-cpp is a skill published in the GitHub repository maystudios/claude-skills (26 stars, last pushed 2mo ago), licensed MIT. It adds 185 tokens to every session and 2,199 once invoked, about $0.0007 per session on Opus 5.5. A static security scan graded it A with 1 finding (makes network calls). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

C and C++Claude Code Data and AI Languages agent-skills ai-coding claude-skills llama-cpp ue5

embedded-stm32

Best practices for embedded C/C++ development on STM32 microcontrollers using the HAL, covering peripherals, DMA, interrupts, memory constraints, and hardware-focused testing. Use when writing STM32 HAL code, configuring peripherals generated by STM32CubeMX, working with interrupts or DMA, debugging with SWD/JTAG…

Mindrally/skills · 87 tokens

llama-cpp