---
language:
- ca
license: apache-2.0
base_model: openai/whisper-large-v3
tags:
- whisper-event
- generated_from_trainer
datasets:
- mozilla-foundation/common_voice_13_0
metrics:
- wer
model-index:
- name: Whisper Large-V3 Catalan
  results:
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: mozilla-foundation/common_voice_13_0 ca
      type: mozilla-foundation/common_voice_13_0
      config: ca
      split: test
      args: ca
    metrics:
    - name: Wer
      type: wer
      value: 5.971420405830237
---

# Whisper Large-V3 Catalan

## Model summary

**Whisper Large-V3 Catalan** is an automatic speech recognition (ASR) model for **Catalan** speech. It is fine-tuned from [openai/whisper-large-v3] on the Catalan portion of **Mozilla Common Voice 13.0**, achieving a **Word Error Rate (WER) of 5.97%** on the Common Voice test split.

The model is intended for high-quality transcription of Catalan speech in a variety of accents and recording conditions, including read and semi-spontaneous speech.

---

## Model description

* **Architecture:** Transformer-based encoder–decoder (Whisper)
* **Base model:** openai/whisper-large-v3
* **Language:** Catalan (ca)
* **Task:** Automatic Speech Recognition (ASR)
* **Output:** Text transcription in Catalan
* **Decoding:** Autoregressive sequence-to-sequence decoding

This model leverages Whisper's multilingual pretraining and large-scale speech-text alignment, followed by supervised fine-tuning on Catalan speech data to improve language-specific accuracy.

---

## Intended use

### Primary use cases

* Transcription of Catalan audio recordings
* Speech-to-text pipelines for media, education, and research
* Accessibility tools (e.g., subtitles, captions)
* Offline or batch ASR for Catalan datasets

### Intended users

* Researchers working on Catalan or low-resource ASR
* Developers building Catalan speech applications
* Institutions and companies requiring Catalan transcription

### Out-of-scope use

* Real-time or low-latency ASR without optimization
* Speech translation (this model performs transcription only)
* Safety-critical applications without additional validation

---

## Limitations and known issues

* Performance may degrade on:

  * Highly noisy audio
  * Strong regional accents underrepresented in Common Voice
  * Conversational or overlapping speech
* The model may produce hallucinated text when audio quality is very poor or silent.
* Biases present in the Common Voice dataset (e.g., demographic or accent imbalance) may be reflected in model outputs.

Users are encouraged to evaluate the model on their own data before deployment.

---

## Training and evaluation data

### Training data

* **Dataset:** Mozilla Common Voice 13.0 (Catalan subset)
* **Data type:** Crowd-sourced, read speech
* **Preprocessing:**

  * Audio resampled to 16 kHz
  * Text normalized using Whisper tokenizer
  * Invalid or excessively long samples filtered

### Evaluation data

* **Dataset:** Common Voice 13.0 (Catalan test split)
* **Metric:** Word Error Rate (WER)

---

## Evaluation results

| Metric     | Value     |
| ---------- | --------- |
| WER (test) | **5.97%** |

These results indicate strong performance compared to the base Whisper multilingual model on Catalan speech.

---

## Training procedure

### Training hyperparameters

* Learning rate: 1e-5
* Optimizer: Adam (β1=0.9, β2=0.999, ε=1e-8)
* LR scheduler: Linear
* Warmup steps: 500
* Training steps: 20,000
* Train batch size: 32
* Gradient accumulation steps: 2
* Effective batch size: 64
* Evaluation batch size: 16
* Mixed precision: FP16 (Native AMP)
* Seed: 42

### Training results (summary)

| Training Loss | Epoch | Step  | Validation Loss | Wer    |
|:-------------:|:-----:|:-----:|:---------------:|:------:|
| 0.0988        | 1.95  | 1000  | 0.1487          | 6.5619 |
| 0.025         | 3.91  | 2000  | 0.1676          | 6.3155 |
| 0.0105        | 5.86  | 3000  | 0.1871          | 6.4035 |
| 0.0047        | 7.81  | 4000  | 0.1973          | 6.4870 |
| 0.0061        | 9.77  | 5000  | 0.2086          | 6.4836 |
| 0.0034        | 11.72 | 6000  | 0.2172          | 6.6442 |
| 0.0036        | 13.67 | 7000  | 0.2205          | 6.4041 |
| 0.002         | 15.62 | 8000  | 0.2214          | 6.4350 |
| 0.0011        | 17.58 | 9000  | 0.2339          | 6.1943 |
| 0.0009        | 19.53 | 10000 | 0.2388          | 6.2921 |
| 0.0011        | 21.48 | 11000 | 0.2327          | 6.2515 |
| 0.0003        | 23.44 | 12000 | 0.2472          | 6.2052 |
| 0.0012        | 25.39 | 13000 | 0.2382          | 6.2892 |
| 0.0001        | 27.34 | 14000 | 0.2550          | 5.9949 |
| 0.0006        | 29.3  | 15000 | 0.2574          | 6.3607 |
| 0.0001        | 31.25 | 16000 | 0.2584          | 6.0143 |
| 0.0001        | 33.2  | 17000 | 0.2686          | 5.9486 |
| 0.0           | 35.16 | 18000 | 0.2736          | 5.9194 |
| 0.0           | 37.11 | 19000 | 0.2768          | 5.9646 |
| 0.0           | 39.06 | 20000 | 0.2783          | 5.9714 |


### Framework versions

- Transformers 4.37.2
- Pytorch 2.2.0+cu121
- Datasets 2.16.1
- Tokenizers 0.15.1

---

## How to use

```python
from transformers import pipeline

hf_model = "HiTZ/whisper-large-v3-ca"
device = 0 # set to -1 for CPU

pipe = pipeline(
    task="automatic-speech-recognition",
    model=hf_model,
    device=device 
)

result = pipe("audio.wav")
print(result["text"])
```

---

## Ethical considerations and risks

* This model transcribes speech and may process personal data.
* Users should ensure compliance with applicable data protection laws (e.g., GDPR).
* The model should not be used for surveillance or non-consensual audio processing.

---

## Citation

If you use this model in your research, please cite:

```bibtex
@misc{dezuazo2025whisperlmimprovingasrmodels,
  title={Whisper-LM: Improving ASR Models with Language Models for Low-Resource Languages},
  author={Xabier de Zuazo and Eva Navas and Ibon Saratxaga and Inma Hernáez Rioja},
  year={2025},
  eprint={2503.23542},
  archivePrefix={arXiv},
  primaryClass={cs.CL}
}
```

Please, check the related paper preprint in
[arXiv:2503.23542](https://arxiv.org/abs/2503.23542)
for more details.

---

## License

This model is available under the
[Apache-2.0 License](https://www.apache.org/licenses/LICENSE-2.0).
You are free to use, modify, and distribute this model as long as you credit
the original creators.

---

## Contact and attribution

* Fine-tuning and evaluation: HiTZ/Aholab - Basque Center for Language Technology
* Base model: OpenAI Whisper
* Dataset: Mozilla Common Voice

For questions or issues, please open an issue in the model repository.