killkli/open-jev-laya-multilingual-onnx

Laya multilingual ONNX for browser decisions

This repository contains a browser-ready ONNX export of the publicly licensed Convai Innovations Laya multilingual checkpoint. The weights remain under Apache 2.0. The export used the receptron/laya ONNX exporter at the linked revision.

The graph has five inputs (input_ids, attention_mask, marker_pos, marker_mask, qtype) and returns decision logits plus act_probs. It runs through the low-level PreTrainedModel.from_pretrained() API in Transformers.js with model_file_name: "laya" and subfolder: "onnx". Use dtype: "fp16" for onnx/laya_fp16.onnx on WebGPU devices with shader-f16 or on the WASM backend; use dtype: "fp32" for onnx/laya.onnx as fallback. Both fp16 paths were tested in headless Chrome. The model does not generate text. The browser sequence encoder and typed answer decoder live in open-jev.

The fp32 export is 1,290,793,955 bytes and the float16 variant is 646,982,318 bytes. The fp16 variant was converted with onnxconverter-common==1.14.0 and onnx==1.17.0; input and output tensor types remain unchanged. In a 21-case, 60-question English/Chinese comparison covering choice, score, and noul questions, fp16 agreed with fp32 on all top choices; maximum absolute choice-probability difference was 0.00245. These examples are regression probes, not a full quality benchmark. Decision probabilities have not been separately recalibrated for the open-jev examples.