nyadla-sys/whisper-tiny.en.tflite

Whisper TFLite Model Generation and Test

Converts OpenAI Whisper speech recognition models to TFLite format for on-device inference (e.g. Android), and generates the mel filter + vocab binary file needed by native C++ runtimes.

Requirements

• Python 3.9
• macOS, Linux, or Google Colab

Dependencies are installed automatically on first run.

Three Generation Modes

The script supports three modes depending on the --language argument:

1. English-only (.en models)

python3.9 whisper_tflite_model_generation_and_test.py --model whisper-tiny.en

• Output: whisper-tiny.en.tflite
• Signature: serving_default (transcribe English)
• forced_decoder_ids: [[2, 50359], [3, 50363]]

2. Single-language (explicit language code)

python3.9 whisper_tflite_model_generation_and_test.py --model whisper-base --language fr
python3.9 whisper_tflite_model_generation_and_test.py --model whisper-base --language de

• Output: whisper-base.fr.tflite, whisper-base.de.tflite
• Signature: serving_default (transcribe the specified language)
• forced_decoder_ids: [[1, <lang_token>], [2, 50359], [3, 50363]]

3. Transcribe-translate (auto language detection)

python3.9 whisper_tflite_model_generation_and_test.py --model whisper-base --language auto

• Output: whisper-base-transcribe-translate.tflite
• Signatures: serving_default (= transcribe), serving_transcribe, serving_translate
• No language token forced — Whisper auto-detects the spoken language
• forced_decoder_ids (transcribe): [[2, 50359], [3, 50363]]
• forced_decoder_ids (translate): [[2, 50358], [3, 50363]]

Usage

# Default: whisper-tiny.en (English-only)
python3.9 whisper_tflite_model_generation_and_test.py

# Single-language French model
python3.9 whisper_tflite_model_generation_and_test.py --model whisper-base --language fr

# Transcribe-translate model (auto-detect language)
python3.9 whisper_tflite_model_generation_and_test.py --model whisper-base --language auto

# Test translate signature specifically
python3.9 whisper_tflite_model_generation_and_test.py --model whisper-small --language auto --task translate

# Show all options
python3.9 whisper_tflite_model_generation_and_test.py --help

Command Line Arguments

Argument	Default	Description
--model	whisper-tiny.en	Whisper model to convert
--language	en	Language code (en, fr, de, ...) or auto for auto-detection
--task	transcribe	transcribe or translate. Only relevant with --language auto
--multilingual / --no-multilingual	--multilingual	Vocab binary type. Auto-set to --no-multilingual for .en models

Supported Models

Model	Type	Parameters	Required VRAM	Relative Speed
whisper-tiny.en	English-only	~39M	~1 GB	~10x
whisper-tiny	Multilingual	~39M	~1 GB	~10x
whisper-base.en	English-only	~74M	~1 GB	~7x
whisper-base	Multilingual	~74M	~1 GB	~7x
whisper-small.en	English-only	~244M	~2 GB	~4x
whisper-small	Multilingual	~244M	~2 GB	~4x
whisper-medium.en	English-only	~769M	~5 GB	~2x
whisper-medium	Multilingual	~769M	~5 GB	~2x
whisper-large	Multilingual	~1550M	~10 GB	1x
whisper-large-v3	Multilingual	~1550M	~10 GB	1x
whisper-turbo	Multilingual	~809M	~6 GB	~8x

Supported Languages

en, fr, hi, ko, de, zh, ja, es, ar, ru, pt, it, nl, sv, pl, da, fi, and many more.

Use auto for language auto-detection. See the full list.

TFLite Serving Signatures

Mode	Signatures	Description
English-only (.en)	serving_default	Transcribe English
Single-language	serving_default	Transcribe the forced language
Transcribe-translate (auto)	serving_default, serving_transcribe, serving_translate	Auto-detect language, transcribe or translate to English

Token reference:

• 50358 = <|translate|>
• 50359 = <|transcribe|>
• 50363 = <|notimestamps|>

What the Script Does

Step	Description
0	Install/verify Python dependencies
1	Configure model parameters and fetch decoder token mappings
2	Load the Whisper model and run a test transcription (English: LibriSpeech, other languages: Google FLEURS)
3	Patch TFForceTokensLogitsProcessor to avoid NaN values during TFLite export
4	Wrap the model with serving signature(s) and save as TF SavedModel
5	Convert the SavedModel to TFLite with dynamic range quantization
6	Verify the TFLite model produces correct output via the TFLite Interpreter
7	(Optional) Test the TFLite model against .wav audio files
8	Generate the mel filters + vocab binary file

Output Files

# English-only
whisper-tiny.en.tflite
filters_vocab_en.bin

# Single-language (e.g. French)
whisper-base.fr.tflite
filters_vocab_multilingual.bin

# Transcribe-translate (auto)
whisper-base-transcribe-translate.tflite
filters_vocab_multilingual.bin

These files are what you need for on-device Whisper inference on Android or other embedded platforms.

Citing

If you are using the Whisper tflite model, please cite:

@misc{nyadla-sys,
  author={Niranjan Yadla},
  title={{Whisper TFLite: OpenAI Whisper Model Port for Edge Devices}},
  year=2022,
  howpublished={GitHub Repository},
  url={https://github.com/nyadla-sys/whisper.tflite},
  url={https://github.com/moonshine-ai/openai-whisper},
  note={Original TFLite implementation of OpenAI Whisper for on-device automatic speech recognition}
}