Koyal Malayalam 120M — unofficial Core ML conversion

This repository contains unofficial FP32 and FP16 Core ML format conversions of Adalat AI's Koyal Malayalam ASR 120M. The original model and learned weights are by Adalat AI. matjacob prepared the conversion, reference scripts, and validation. No additional training, fine-tuning was performed. The root packages retain FP32; fp16/ reduces weights and intermediate computation to FP16. No integer quantization was performed. This release is not endorsed by or affiliated with Adalat AI.

The model weights remain under CC-BY-4.0. See LICENSE, NOTICE, and the preserved upstream model card. The original encoder was initialized from AI4Bharat's MIT-licensed MahaDhwani checkpoint; see the upstream card for architecture, training, citations, and original evaluation results.

What is included

Choose a precision: the repository root contains the original FP32 packages (about 449 MiB); fp16/ contains the smaller FP16 packages (about 225 MiB), with its own manifest, tokenizer, test vectors, and validation reports. FP16 is intended as an option for device integration; FP32 remains available for comparison.

Artifact Purpose
encoder.mlpackage Conformer encoder: fixed [1, 80, 512] padded mel input plus valid length
predictor.mlpackage Stateful single-token TDT prediction network
joint.mlpackage Joint token and duration scores for one encoder/predictor step
tokenizer.model, vocabulary.json Original SentencePiece tokenizer and token vocabulary
manifest.json Input/output shapes, preprocessing settings, provenance, validation flags
*-test-inputs.npz, *-test-outputs.npz Synthetic PyTorch component test vectors
validation.json Core ML versus PyTorch numerical comparison
transcription-check.json Three local clip transcription comparisons; audio is not redistributed
export-koyal-coreml.py Direct NeMo/PyTorch to Core ML export
validate-koyal-coreml.py Core ML prediction checks against the test vectors
transcribe-koyal-coreml.py Short-clip reference inference and optional NeMo comparison

The models use ML Program, batch size one, and fixed mel-frame input. The deployment target is the macOS 14 / iOS 17 generation. An application must supply feature extraction, padding, SentencePiece text decoding, and the TDT blank/duration loop. This repository supplies a Python reference for that loop, not a complete Swift application.

Download and try on macOS

Use a Python 3.10 environment with the dependencies in requirements.txt and ffmpeg available on PATH. The conversion was tested with coremltools 9.0, NeMo 3.0.0, and PyTorch 2.14.1. coremltools warns that this PyTorch version is outside its tested version range; the validation results below were obtained with this installed combination.

from huggingface_hub import snapshot_download
snapshot_download("matjacob/koyal-ml-120m-1.0-coreml", local_dir="koyal-coreml")

For the FP16 models and reference runtime without downloading FP32 weights:

from huggingface_hub import snapshot_download
snapshot_download(
    "matjacob/koyal-ml-120m-1.0-coreml",
    local_dir="koyal-coreml",
    allow_patterns=["fp16/*", "*.py", "requirements.txt", "README.md", "LICENSE", "NOTICE"],
)
cd koyal-coreml
python validate-koyal-coreml.py .
python transcribe-koyal-coreml.py \
  --models . --audio /path/to/short-clip.wav --output result.json

Select FP16 with --models ./fp16. To allow Neural Engine execution with CPU fallback, add --compute-units cpu_and_ne; the default remains cpu_only. --compute-units all also allows GPU execution. Choosing a compute-unit policy does not establish which operations actually run on each device.

The reference accepts audio readable by ffmpeg and resamples to 16 kHz mono. It uses Core ML CPU_ONLY for all three neural components and NeMo/PyTorch only for matching mel preprocessing. Its preprocessing disables dither for deterministic inference. The optional --compare-nemo /path/to/checkpoint.nemo also runs the original model; the original gated checkpoint must be obtained from its upstream repository separately.

512 mel frames accommodate roughly 5.1 seconds of audio. The script rejects clips whose mel features exceed that size. For longer recordings, chunk the audio or make a separately validated export with a larger fixed frame count. The export script defaults to 1024 frames; pass --frames 512 to reproduce the released models.

Conversion and provenance

Source: adalat-ai/koyal-ml-120m-1.0 at revision 8135bc822466bf70d75c238041c585040b56e9f8.

Source checkpoint SHA-256: 96a69cab106eee4075c3fdae1e32e75f8590467ed8c6abcf91f3bf039b0c3c4e.

Changes from upstream: exported the encoder, prediction network, and joint network as separate Core ML packages; disabled the joint's training-only fused loss/WER path for export; traced a fixed 512-frame encoder and single-token decoder/joint inputs; retained FP32 computation and original tokenizer; added reference inference, manifests, attribution, and test vectors. The optional fp16/ conversion reduces weight and intermediate computation precision; its external float inputs/outputs retain the exported interface. The auxiliary CTC head is not exported. This release uses TDT decoding.

Reproduce on macOS after obtaining the upstream checkpoint:

python export-koyal-coreml.py \
  --model /path/to/koyal-ml-120m-1.0.nemo \
  --output-dir ./converted --frames 512
python validate-koyal-coreml.py ./converted

To reproduce FP16, add --fp16 to the export command and use a separate output directory. The strict numerical validator exits nonzero for this FP16 encoder; the recorded tolerance failure is described below rather than suppressed.

Validation and limits

Core ML CPU predictions were compared with PyTorch synthetic test vectors:

Component Maximum absolute output difference
Encoder 0.00001480
Predictor (including recurrent states) 0.00000009
Joint 0.00001908

Three short Malayalam vocabulary clips produced identical complete transcripts with the Core ML reference pipeline and original NeMo model: ഉണ്ട് ഉണ്ട്, ഇന്ത്യൻ എക്സ്പ്രസ് ഇന്ത്യൻ എക്സ്പ്രസ്, and കൈകൂപ്പി കൈകൂപ്പി. This is a small conversion sanity check, not an accuracy benchmark or a claim that the original model's published WER carries over to all Core ML uses.

FP16 validation

FP32 was additionally checked against original NeMo inference on 15 clips: all transcripts matched exactly. The clip sample spans multiple lessons and includes both previously accepted and corrected transcript labels. One longer clip from the earlier 16-clip ASR sample was excluded because it exceeds the 512-frame input limit.

FP16 execution policy Exact transcript matches against FP32
CPU_ONLY 15 / 15
CPU_AND_NE 14 / 15

The remaining CPU_AND_NE result differs only in whitespace: കൈകൂപ്പി കൈകൂപ്പി versus കൈകൂപ്പി കൈ കൂപ്പി. CPU_ONLY token start times shifted by up to 0.08 seconds on these clips. CPU_AND_NE permits CPU fallback; this test did not inspect operation placement and is not proof that the model ran wholly or partly on the Neural Engine. Raw token positions and transcripts are included in fp16/*-transcriptions.json; the summarized comparison is fp16/transcription-comparison.json.

Synthetic component comparisons against PyTorch FP32 produced maximum absolute differences of 0.114132 for encoder features, 0.000547 for predictor states, and 0.136650 for joint scores. The encoder fails the validator's strict elementwise rtol=0.005, atol=0.005 check. The predictor and joint pass it (relative tolerance applies to the joint's larger-valued scores). These results are retained in fp16/validation.json; the encoder manifest flag remains false. FP16 is therefore supported by small-sample text comparisons, not full numerical equivalence or a broad accuracy benchmark.

iPhone performance, Swift integration, verified Neural Engine placement, GPU execution, integer quantization, long-recording chunking, and alternate frame sizes have not been validated. Core ML format alone does not guarantee Neural Engine execution. Token positions in reference output use the encoder's 40 ms step; TDT duration predictions are frame skips, not verified phonetic word endings. Upstream limitations apply, including Malayalam-only recognition and weaker handling of code-switching or speech outside the training domain.

Attribution

Please credit Adalat AI and link the original Koyal model when using or redistributing these weights, retain the CC-BY-4.0 license, and identify this release as an unofficial Core ML conversion by matjacob. Preserve the source attribution and indicate your own changes when modifying this release. Original research citations are retained in UPSTREAM_MODEL_CARD.md.

Downloads last month
8
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for matjacob/koyal-ml-120m-1.0-coreml

Quantized
(2)
this model