litert-community
/

LFM2.5-1.2B-JP

@@ -68,7 +68,7 @@ Install a recent [Google AI Edge Gallery](https://github.com/google-ai-edge/gall
 Set `--max-num-tokens` to the smallest value your use case needs — 1024 is a good chat default, and the file allows up to 4096. At 1024 the int4 file decodes fastest; at 4096 the two variants converge.
-**Use the CPU backend.** Asking `litert-lm benchmark` for the GPU backend fails at engine creation on this Mac with `Hint fully delegated to single delegate is set, but the graph is not fully delegated` — the hybrid short-convolution blocks do not fully lower to the Metal delegate, so there is no GPU row. A Metal engine-creation failure for this family is tracked upstream in [LiteRT-LM#3129](https://github.com/google-ai-edge/LiteRT-LM/issues/3129).
 On a Pixel 8a (Tensor G3, CPU backend, measured in AI Edge Gallery) int8 decodes at ~19 tok/s and int4 at ~31 tok/s — on phone-class memory bandwidth the int4 file is about 1.7× faster as well as 41% smaller, so prefer int4 on mid-range devices. Those Android figures are single ship-gate runs, not medians. First device load compiles the graph and can take about a minute; later loads are instant.

 Set `--max-num-tokens` to the smallest value your use case needs — 1024 is a good chat default, and the file allows up to 4096. At 1024 the int4 file decodes fastest; at 4096 the two variants converge.
+**Use the CPU backend — this bundle cannot create a GPU engine.** It comes from the pre-0.9.2 ShortConv export generation, whose prefill graph still carries INT64 `ADD`/`CAST` inside `Lfm2ShortConv`, plus `GATHER_ND` and a `GREATER_EQUAL` with const inputs. The GPU delegate takes 536 of the 579 operations and leaves 43 on the CPU, and the runtime then refuses the partial split: `Hint fully delegated to single delegate is set, but the graph is not fully delegated`. Re-exporting from the post-0.9.2 lineage removes those INT64 ops and does run fully delegated on the macOS GPU; the remaining iOS Metal failure is tracked upstream in [LiteRT-LM#3129](https://github.com/google-ai-edge/LiteRT-LM/issues/3129).
 On a Pixel 8a (Tensor G3, CPU backend, measured in AI Edge Gallery) int8 decodes at ~19 tok/s and int4 at ~31 tok/s — on phone-class memory bandwidth the int4 file is about 1.7× faster as well as 41% smaller, so prefer int4 on mid-range devices. Those Android figures are single ship-gate runs, not medians. First device load compiles the graph and can take about a minute; later loads are instant.