Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add google-ai-edge/litert-samples --skill gpu-clean-conversiongit clone --depth 1 https://github.com/google-ai-edge/litert-samplesWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/google-ai-edge/litert-samples/gpu-clean-conversion)<a href="https://agentmods.dev/skills/google-ai-edge/litert-samples/gpu-clean-conversion"><img src="https://agentmods.dev/badge/skills/google-ai-edge/litert-samples/gpu-clean-conversion/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/google-ai-edge/litert-samples/gpu-clean-conversion"><img src="https://agentmods.dev/badge/skills/google-ai-edge/litert-samples/gpu-clean-conversion.svg" alt="Reviewed on agentmods" width="80" height="20"></a>- NVIDIA SkillSpector warn
SkillSpector: 1 finding, up to medium
These are SkillSpector’s own severities. On a checked sample its high-severity flags on skills were ~96% false positives — a documented command, a public API, a “never do X” rule — so we show them as a caution to read, not a verdict. Why →
- medium analysis-evasion · line 1 Suspicious Unicode normalization or mixed-script contentFix: Review the flagged content for security risks. Ensure no credentials, secrets, or sensitive data are exposed.
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00076 | $0.03140 |
| Opus 5.5 | $0.00030 | $0.01256 |
| Sonnet 5.5 | $0.00015 | $0.00628 |
| Haiku 4.5 | $0.00008 | $0.00314 |
Grade A, and why
gpu-clean-conversion scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 14d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 130 lines — stays where its author put it; the contents beside it link to each section on GitHub.
GPU-clean conversion
A conversion is done when three things hold, in this order:
- it converts,
- every node runs on the GPU,
- the on-device output matches the source model.
Step 3 is not implied by step 2. The delegate can report full residency and still return wrong numbers — that is the failure mode most of the rewrites below exist for. Never call a model done without a numerical check against CPU or PyTorch on the actual device.
Loop
1. Convert plain first. No patches. This tells you what the model actually needs rather than what you assumed.
2. Verify through the CompiledModel API before touching a device.
from litert_gpu_toolkit import check_gpu_compatibility, print_report
print_report(check_gpu_compatibility("model.tflite"))
It compiles the model for the GPU accelerator, runs every signature on random inputs, and compares the outputs against a CPU-compiled reference. A failed GPU compile names the offending op — route it through the table below. A pass with a CPU-fallback warning means some ops fell back to the CPU; treat them the same way if you need full residency. This exercises the host GPU, so step 5 on the actual device is still required.
3. Map each symptom to a rewrite. The rewrites live in
utilities/litert_gpu_toolkit — plain Python, no build step. Outside this
repo, clone litert-samples and put utilities/ on PYTHONPATH, or vendor
the directory:
| What you see | Rewrite |
|---|---|
GATHER_ND named in the compile error |
Find its source. Stride-2 slicing (x[:, ::2], Focus stems, patch merging), grid_sample, MaxPool padding, bicubic interpolate, and reflect-mode F.pad all lower to it. patch_grid_sample, patch_maxpool_zeropad, patch_interpolate, patch_patch_merging |
| A rank-5+ tensor named in the compile error | PixelShuffle (rank-6 reshape), windowed attention, einops.rearrange, packed-QKV attention head splits. pixelshuffle_to_conv_transpose, patch_window_attention, patch_einops |
TRANSPOSE_CONV rejected |
Version skew, not a missing op. ZeroStuffConvT1d / ZeroStuffConvT2d — zero-stuff plus a plain conv, exact to ~1e-7 |
SELECT / SELECT_V2 |
PReLU, ELU, in-place index assignment, and torch.where masking. Replace with arithmetic: x*(1-m) + v*m. Keep v finite in fp16: −1e9 is −inf there, and (1 − m) × −1e9 is 0 × −inf = NaN on every valid position; −30000 is exact in fp16 and gives the same softmax in fp32 when the logits are bounded |
BROADCAST_TO |
Two cases. On a compile-time constant: an outer product or .expand that did not fold — bake the result as a constant at its target shape. On a runtime tensor the GPU delegate rejects it outright, even at rank 4 — the canonical case is GQA's repeat_kv (x[:,:,None].expand(...), which is also rank-5, so two walls in one line). Exact rewrite: torch.cat([x[:, i:i+1].expand(b, n_rep, s, d) for i in range(n_kv)], dim=1) — same head order, bit-exact. Tracked upstream: google-ai-edge/LiteRT#9191 |
| Masked attention wrong only on device: token 0 bit-exact, every later token wrong | Broadcast ADD whose LHS is a BATCH_MATMUL result (the scores + mask[1,1,S,S] idiom) silently miscomputed on ai-edge-litert 2.1.3 and 2.1.5 and is fixed in 2.1.6 (head-axis size-1 broadcast only; google-ai-edge/LiteRT#8593, closed as fixed; the 2.1.6 release notes list GPU broadcast fixes). The signature mimics broken RoPE — tap the rope output before blaming it. Rewrites on the older runtimes: pre-expand the mask to [1,H,S,S], or materialize the BMM as a second output |
An ADD result that is both a graph output and consumed downstream comes back wrong |
Output aliasing: the returned tensor holds an operand, not the sum — ADD with two runtime operands (SUB/MUL exact, x + 1.0 exact). This is the shape of every explicit state update in a streaming/recurrent graph. Workaround: emit acc * one where one is a runtime input holding 1.0 — a constant folds straight back into the pattern. Tracked upstream: google-ai-edge/LiteRT#8599 |
RELU_0_TO_1 rejected by the GPU delegate |
Emitted by hard-sigmoid / nn.Hardtanh(0,1). Accepted in litert 2.1.3, rejected from 2.1.5 on — a model at full residency on an older runtime hard-fails CompiledModel creation after an upgrade. Rewrite: relu(x) - relu(x-1), exact. Tracked upstream: google-ai-edge/LiteRT#8598 |
DIV: No support of few identical inputs / Expected 1 const input tensor(s), device only |
The delegate declines an op whose two inputs are the same tensor, and ops whose inputs are all constants — together these split a perceiver-style block (softmax over a length-1 axis of a constant latent bank) into several partitions. Fixes: special-case the degenerate axis (a softmax over a length-1 axis is identically 1), or make one input non-constant. Note the sibling LayerNorm-over-a-constant pattern no longer reaches the delegate at all — the converter folds it to a single MUL. Tracked upstream: google-ai-edge/LiteRT#9192 |
NHWC node rewriter not found: amax |
x.amax(...)/x.max(dim) in stable-softmax, adaptive norms, qk-norm. Rewrite channel reduce-max as max_pool2d(x.reshape(N,1,C,H*W), kernel=(C,1)) — numerically identical — or drop the norm to 3D |
Lowering not found: aten._fft_r2c / aten.complex |
torch.stft/istft and complex views have no lowering (fails before any GPU check). A DFT is a fixed linear map: windowed-DFT as Conv1d with the cos/sin basis baked into kernels (stride = hop), iSTFT as inverse-DFT matmul + overlap-add via zero-stuffed conv-transpose — exact. Model-selection corollary: prefer time-domain vocoder branches over iSTFT-based ones. ⚠ Library STFT-as-conv stacks (torchlibrosa-style) have numerically mis-converted (corr 0.83) while the op check looks clean — verify the spectrogram numerically or compute log-mel host-side |
| Compiles, runs, output is wrong or NaN | The fp16 reduction family — and the trigger is the fp16 accumulator passing 65504, so a plain single-axis mean/sum over enough elements overflows just like variance does. patch_safe_layernorm, patch_rmsnorm, patch_instance_norm, hierarchical_mean. Caveats: at extreme magnitudes (|x| in the thousands) even the adaptive safe-LN form overflows when it reconstructs the large variance — the robust form stays entirely in the down-scaled domain (xs = x/S, normalize xs, never multiply the variance back by S²); hierarchical_mean is exact only for power-of-two spatial dims (for arbitrary dims, cascade /2 avg-pools with ceil_mode so each stage averages ≤~49 elements). A static form keeps fp32 bit-exact and puts no reduce-max in the graph: scale each norm's input by a power of two 2^-k chosen per site from a calibration over the whole check set (headroom for the row's sum of squares and for the largest single x² — both overflowed in the case on record, the mean of squares did not), and eps by 4^-k. When eps·4^-k drops out of fp16's range (eps 1e-6 at k = 3), an all-zero row (padding) becomes rsqrt(0) = inf and 0 × inf = NaN: clamp the rsqrt at 2^k/√eps — the zero row's own value, never reached in fp32. Diagnostic split: all-zero/all-blank output = an overflow in one block; a result that starts near-correct and degrades with depth = precision compounding, which no overflow patch (and no fp32-precision flag) fixes |
| Head outputs exactly zero | RMSNorm Σx² overflowed fp16 to inf. patch_rmsnorm |
GPU compile fails only at fp16 precision with a kernel-source error (write_imageh on Adreno and Mali OpenCL, write on Metal, at the same source line), while fp32 precision compiles |
A boolean value on the way to a graph output. In the case on record an output mask computed as (mask > 0.5) and cast back to float made all three GPUs fail; the same output computed from the float input mask by float ops (a strided slice) compiled at fp16 everywhere and stayed bit-exact in fp32. Small graphs with the same GREATER → CAST pair compiled, so bisect on the real graph. Keep masks float (1 = valid) from input to output |
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 14d ago Changed · +1 lines 9623b051112f
- 27d ago Changed · +7 lines 996e2c7eb940
- 2mo ago First seen · 122 lines · 76 tokens per session scan A 08bdb7b8afb2
gpu-clean-conversion is a skill published in the GitHub repository google-ai-edge/litert-samples (486 stars, last pushed yesterday), licensed Apache-2.0. It adds 76 tokens to every session and 3,140 once invoked, about $0.0003 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
adversarial-ml-evasion
Craft adversarial examples that cause trained ML classifiers to misclassify at inference time — image recognition, malware detectors, IDS, spam filters.
llm-redteam-overview
LLM red team category — full AATMF v3 tactic coverage (T01–T15). Routing skill: read this first to identify which tactic applies, then load the matching sub-skill. Maps to MITRE ATLAS where overlap exists.
misinformation
Hunt LLM misinformation / overreliance (OWASP LLM09:2025) — confident-but-wrong outputs that flow into downstream automated decisions, compliance reports, customer communications, or autonomous code commits without verification.
sensitive-information-disclosure
Hunt LLM sensitive-information disclosure (OWASP LLM02:2025) — leakage of PII, secrets, internal source, model details, and other-tenant data through model outputs, training-data extraction, or retrieval-side joins.
vector-and-embedding-weaknesses
Hunt vector / embedding weaknesses (OWASP LLM08:2025) — adversarial inputs against the RAG / similarity layer that cause cross-tenant leak, embedding-inversion privacy loss, semantic confusion, and retriever-driven prompt injection.
aatmf-t12-rag-poisoning
AATMF T12 — RAG & Knowledge Base Manipulation. PoisonedRAG, vector store flood, embedding collision, retrieval-bias attacks.