Torick63/opus-mt-en-es-onnx-int8

opus-mt-en-es, ONNX int8 (merged decoder)

Helsinki-NLP/opus-mt-en-es (revision 5bc4493), exported to ONNX for on-device translation.

• Exported with Optimum (--task text2text-generation-with-past): one encoder and one merged decoder that handles both the first step and the cached steps (use_cache_branch input), so the decoder weights are stored once.
• Dynamic int8 weight quantisation (ONNX Runtime quantize_dynamic, EnableSubgraph so the layers inside the merged decoder's branches are quantised too).
• About 149 MB in total.

File	Purpose
encoder_model.onnx	Encoder
decoder_model_merged.onnx	Decoder with KV cache
source.spm, target.spm	SentencePiece models
vocab.json	Shared vocabulary
config.json, generation_config.json	Token IDs (eos, pad, decoder start) and generation settings

With greedy decoding, the int8 files give the same output as the original PyTorch model on 26 of 30 short test phrases; the rest mostly differ in wording (quantisation).

Licence and credit

The model is by the Language Technology Research Group at the University of Helsinki (OPUS-MT, Tiedemann & Thottingal, 2020), released under Apache-2.0; this conversion is released under the same licence.