jev-audit

Inference-cost audit

Most production LLM calls return a label, not prose — a route, a score, a yes/no, a category. Those calls pay for autoregressive generation they do not use. This audit finds them and prices the swap.

Deliverable: a ranked table of candidates with measured deltas. Sellable as a OneWave engagement; same shape as marketing-brain — a repeatable audit run per client.

1. Find the candidates

Search for LLM calls, then filter to the ones whose output is a label:

grep -rnE "messages\.create|chat\.completions|generateText|\.invoke\(|anthropic\.|openai\." \
  --include=*.{ts,tsx,js,py} . | grep -v node_modules

A call is a candidate when all of these hold:

• the prompt asks for one of a known, finite set of answers (≤255)
• the caller parses the response — JSON.parse, a regex, an enum lookup, .trim()
• nothing downstream shows a human the model's reasoning
• it runs often, or a user waits on it

Strong signals in the prompt text: "respond with only", "return JSON", "classify", "choose one of", "rate from 1 to", "answer yes or no", "do not explain".

Disqualifiers: the output is shown to a user, is used as content, needs a citation or justification, or the answer set is open-ended.

2. Measure what it costs today

Do not estimate. Instrument:

• calls/day — from logs or a counter, not a guess
• p50 and p95 latency — p95 is what users feel
• tokens in/out per call → current $/1k calls at the provider's list price
• is anything blocked on it — a user, a page render, an agent's next step

3. Price the swap

Benchmark against a labelled set from that call's real traffic (see jev-eval). Never project from a vendor benchmark.

Reference numbers, measured on real records, 4 questions per record:

	latency	cost/1k	accuracy
Jev hosted	399 ms	$0.023	matched Claude on a 15-record set
Von local	174 ms	$0.00	matched only with well-written criteria
Claude Haiku 4.5	2,322 ms	~$0.55	baseline