Canary 1B v2

NVIDIA NeMo's Canary 1B v2 speech recognition and translation model (25 languages), exported for loom.cpp.

This is a loom.cpp export: a single self-describing GGUF that carries its own graph topologies, tokenizer (if any) and driver script, produced by loom-exporter.

Original model

Exported from nvidia/canary-1b-v2. Weights are unmodified; this repo packages the same parameters into loom.cpp's GGUF format.

License

cc-by-4.0, inherited from the base model above.

Language(s)

bg, hr, cs, da, nl, en, et, fi, fr, de, el, hu, it, lv, lt, mt, pl, pt, ro, sk, sl, es, sv, ru, uk

Usage

Run it with loom-py -- loom-py-rt on PyPI:

pip install -U "loom-py-rt[hub]"

Needs loom-py-rt 1.0.0rc18 or later. This file uses the engine's STFT op, new in that release; an older runtime stops with unknown op 'STFT'.

import loom

model = loom.Model.from_pretrained("loom-ai-org/canary-1b-v2-loom")

# Audio is a mono float list at 16 kHz. Past 40 s, what the model was trained on, it is decoded in
# overlapping windows and stitched, as NeMo does (see Known limitations for the version this needs).
# `language` is what the audio is in; it is not detected, and defaults to English.
result = model.speech2text.infer(audio)
print(result.text)

# `target_language` is what to WRITE, and it defaults to English: German audio with no target comes
# back translated, and language="de", target_language="de" keeps it German.
print(model.speech2text.infer(audio, language="en", target_language="fr").text)

The layer underneath

The call above is the high-level door: one per task, named for the modality pair it maps between, with the windowing, sampling and assembly this model needs already applied. Under it, model.infer(...) passes your arguments straight to the driver this GGUF embeds -- which is where you go for a knob the door does not name.

model.driver_source prints that driver, including a header comment documenting every argument it accepts for this model, and is the authority on it. See loom-py for the API and loom.cpp for what the engine does between the two.

Known limitations

Long audio needs loom 1.0.0-rc14 or later. The model was trained on clips of up to 40 s. Past that, this file tells loom to do what NeMo's own transcribe() does: cut the audio into overlapping 30-40 s windows, decode each, and stitch the transcripts where they overlap. On 1.0.0-rc13 and earlier a long clip is decoded in one pass, which degrades fast -- on 79 s of LibriSpeech it returned 116 of 181 words.

No timestamps: NeMo gets them from a second (CTC) model inside the .nemo, which is not exported. segments is one span covering the whole clip.

Files

  • canary-1b-v2.gguf -- the model, exported with loom-exporter.
Downloads last month
136
GGUF
Model size
1.0B params
Architecture
loom-canary
Hardware compatibility
Log In to add your hardware

We're not able to determine the quantization variants.

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for loom-ai-org/canary-1b-v2-loom

Quantized
(35)
this model

Collection including loom-ai-org/canary-1b-v2-loom