Koyal Malayalam 120M — unofficial Core ML conversion
This repository contains unofficial FP32 and FP16 Core ML format conversions of
Adalat AI's Koyal Malayalam ASR 120M.
The original model and learned weights are by Adalat AI. matjacob prepared
the conversion, reference scripts, and validation. No additional training,
fine-tuning was performed. The root packages retain FP32; fp16/ reduces
weights and intermediate computation to FP16. No integer quantization was
performed. This release is not endorsed by or
affiliated with Adalat AI.
The model weights remain under CC-BY-4.0. See LICENSE, NOTICE, and the preserved upstream model card. The original encoder was initialized from AI4Bharat's MIT-licensed MahaDhwani checkpoint; see the upstream card for architecture, training, citations, and original evaluation results.
What is included
Choose a precision: the repository root contains the original FP32
packages (about 449 MiB); fp16/ contains the smaller FP16 packages
(about 225 MiB), with its own manifest, tokenizer, test vectors, and validation
reports. FP16 is intended as an option for device integration; FP32 remains
available for comparison.
| Artifact | Purpose |
|---|---|
encoder.mlpackage |
Conformer encoder: fixed [1, 80, 512] padded mel input plus valid length |
predictor.mlpackage |
Stateful single-token TDT prediction network |
joint.mlpackage |
Joint token and duration scores for one encoder/predictor step |
tokenizer.model, vocabulary.json |
Original SentencePiece tokenizer and token vocabulary |
manifest.json |
Input/output shapes, preprocessing settings, provenance, validation flags |
*-test-inputs.npz, *-test-outputs.npz |
Synthetic PyTorch component test vectors |
validation.json |
Core ML versus PyTorch numerical comparison |
transcription-check.json |
Three local clip transcription comparisons; audio is not redistributed |
export-koyal-coreml.py |
Direct NeMo/PyTorch to Core ML export |
validate-koyal-coreml.py |
Core ML prediction checks against the test vectors |
transcribe-koyal-coreml.py |
Short-clip reference inference and optional NeMo comparison |
The models use ML Program, batch size one, and fixed mel-frame input. The deployment target is the macOS 14 / iOS 17 generation. An application must supply feature extraction, padding, SentencePiece text decoding, and the TDT blank/duration loop. This repository supplies a Python reference for that loop, not a complete Swift application.
Download and try on macOS
Use a Python 3.10 environment with the dependencies in requirements.txt and
ffmpeg available on PATH. The conversion was tested with coremltools 9.0,
NeMo 3.0.0, and PyTorch 2.14.1. coremltools warns that this PyTorch version is
outside its tested version range; the validation results below were obtained
with this installed combination.
from huggingface_hub import snapshot_download
snapshot_download("matjacob/koyal-ml-120m-1.0-coreml", local_dir="koyal-coreml")
For the FP16 models and reference runtime without downloading FP32 weights:
from huggingface_hub import snapshot_download
snapshot_download(
"matjacob/koyal-ml-120m-1.0-coreml",
local_dir="koyal-coreml",
allow_patterns=["fp16/*", "*.py", "requirements.txt", "README.md", "LICENSE", "NOTICE"],
)
cd koyal-coreml
python validate-koyal-coreml.py .
python transcribe-koyal-coreml.py \
--models . --audio /path/to/short-clip.wav --output result.json
Select FP16 with --models ./fp16. To allow Neural Engine execution with CPU
fallback, add --compute-units cpu_and_ne; the default remains cpu_only.
--compute-units all also allows GPU execution. Choosing a compute-unit policy
does not establish which operations actually run on each device.
The reference accepts audio readable by ffmpeg and resamples to 16 kHz mono.
It uses Core ML CPU_ONLY for all three neural components and NeMo/PyTorch
only for matching mel preprocessing. Its preprocessing disables dither for
deterministic inference. The optional --compare-nemo /path/to/checkpoint.nemo
also runs the original model; the original gated checkpoint must be obtained
from its upstream repository separately.
512 mel frames accommodate roughly 5.1 seconds of audio. The script rejects
clips whose mel features exceed that size. For longer recordings, chunk the
audio or make a separately validated export with a larger fixed frame count.
The export script defaults to 1024 frames; pass --frames 512 to reproduce
the released models.
Conversion and provenance
Source: adalat-ai/koyal-ml-120m-1.0 at revision
8135bc822466bf70d75c238041c585040b56e9f8.
Source checkpoint SHA-256:
96a69cab106eee4075c3fdae1e32e75f8590467ed8c6abcf91f3bf039b0c3c4e.
Changes from upstream: exported the encoder, prediction network, and joint
network as separate Core ML packages; disabled the joint's training-only
fused loss/WER path for export; traced a fixed 512-frame encoder and
single-token decoder/joint inputs; retained FP32 computation and original
tokenizer; added reference inference, manifests, attribution, and test vectors.
The optional fp16/ conversion reduces weight and intermediate computation
precision; its external float inputs/outputs retain the exported interface.
The auxiliary CTC head is not exported. This release uses TDT decoding.
Reproduce on macOS after obtaining the upstream checkpoint:
python export-koyal-coreml.py \
--model /path/to/koyal-ml-120m-1.0.nemo \
--output-dir ./converted --frames 512
python validate-koyal-coreml.py ./converted
To reproduce FP16, add --fp16 to the export command and use a separate output
directory. The strict numerical validator exits nonzero for this FP16 encoder;
the recorded tolerance failure is described below rather than suppressed.
Validation and limits
Core ML CPU predictions were compared with PyTorch synthetic test vectors:
| Component | Maximum absolute output difference |
|---|---|
| Encoder | 0.00001480 |
| Predictor (including recurrent states) | 0.00000009 |
| Joint | 0.00001908 |
Three short Malayalam vocabulary clips produced identical complete
transcripts with the Core ML reference pipeline and original NeMo model:
ഉണ്ട് ഉണ്ട്, ഇന്ത്യൻ എക്സ്പ്രസ് ഇന്ത്യൻ എക്സ്പ്രസ്, and കൈകൂപ്പി കൈകൂപ്പി.
This is a small conversion sanity check, not an accuracy benchmark or a claim
that the original model's published WER carries over to all Core ML uses.
FP16 validation
FP32 was additionally checked against original NeMo inference on 15 clips: all transcripts matched exactly. The clip sample spans multiple lessons and includes both previously accepted and corrected transcript labels. One longer clip from the earlier 16-clip ASR sample was excluded because it exceeds the 512-frame input limit.
| FP16 execution policy | Exact transcript matches against FP32 |
|---|---|
| CPU_ONLY | 15 / 15 |
| CPU_AND_NE | 14 / 15 |
The remaining CPU_AND_NE result differs only in whitespace:
കൈകൂപ്പി കൈകൂപ്പി versus കൈകൂപ്പി കൈ കൂപ്പി.
CPU_ONLY token start times shifted by up to 0.08 seconds on these clips.
CPU_AND_NE permits CPU fallback; this test did not inspect operation placement
and is not proof that the model ran wholly or partly on the Neural Engine.
Raw token positions and transcripts are included in fp16/*-transcriptions.json;
the summarized comparison is fp16/transcription-comparison.json.
Synthetic component comparisons against PyTorch FP32 produced maximum absolute
differences of 0.114132 for encoder features, 0.000547 for predictor states,
and 0.136650 for joint scores. The encoder fails the validator's strict
elementwise rtol=0.005, atol=0.005 check. The predictor and joint pass it
(relative tolerance applies to the joint's larger-valued scores). These results
are retained in fp16/validation.json; the encoder manifest flag remains false.
FP16 is therefore supported by small-sample text comparisons, not full numerical
equivalence or a broad accuracy benchmark.
iPhone performance, Swift integration, verified Neural Engine placement, GPU execution, integer quantization, long-recording chunking, and alternate frame sizes have not been validated. Core ML format alone does not guarantee Neural Engine execution. Token positions in reference output use the encoder's 40 ms step; TDT duration predictions are frame skips, not verified phonetic word endings. Upstream limitations apply, including Malayalam-only recognition and weaker handling of code-switching or speech outside the training domain.
Attribution
Please credit Adalat AI and link the original Koyal model when using or redistributing these weights, retain the CC-BY-4.0 license, and identify this release as an unofficial Core ML conversion by matjacob. Preserve the source attribution and indicate your own changes when modifying this release. Original research citations are retained in UPSTREAM_MODEL_CARD.md.
- Downloads last month
- 8
Model tree for matjacob/koyal-ml-120m-1.0-coreml
Base model
ai4bharat/MahaDhwani_pretrained_conformer