litert-conversion-workflow

LiteRT-LM conversion workflow

A conversion is done when three things hold, in this order:

1. the bundle loads and generates through the LiteRT-LM engine (not just the raw .tflite graphs),
2. output quality is gated against the source model — a floor gate plus a task-level parity check, not a smoke test,
3. it holds up on the deployment path it claims: the target backend, the target device, and multi-turn conversation.

Each step can pass while the next one fails. A bundle that converts can die at engine creation; an engine that generates can be quantization garbage; a model that answers 8/8 single-turn can crash on message two. The gates exist because every one of these has happened.

Scope: models producing a .litertlm bundle consumed by the LiteRT-LM engine (pip install litert-lm) — text LLMs and vision-language models (a VLM bundle is an LLM bundle plus two vision graphs; references/vlm-conversion.md covers the delta). Standalone/classic .tflite models go through the gpu-clean-conversion → accuracy-safe-quantization → on-device-verification lane; this skill is the LM sibling and reuses their discipline where it applies.

Step 0: classify before you convert

Look at config.json (model_type, layer_types, MoE fields, size) before running anything. Architecture decides everything downstream, and some structures die at a known point no recipe can route around. references/architecture-walls.md is the lookup table: structure → where it dies (export / load / engine / backend) → the error signature you'll see. Check it first; it turns a day of debugging into a table lookup.

Then pick the lane in references/recipe-selector.md: plain dense decoders ride a standard export; hybrids (SSM / linear-attention / short-conv) need state-aware export plus executor metadata; reasoning models need template care; MoE and MLA are currently walls.

Loop

1. Export with a minimal, extraction-safe template. The single most common ship-killer is not math — it is the chat template. Export with use_jinja_template=False and a minimal ChatML-style template swapped in, so the bundle carries plain prefix/suffix markers and no Jinja at all. Vendor templates routinely call Python methods (.get(), .startswith(), .strip()) that the runtime's minijinja renderer does not implement — such a bundle imports fine and dies on the first message. Details, the multi-turn prefix contract, and the tokenizer traps that pair with this: references/template-tokenizer-traps.md.