Kokoro-Thai INT8 (Refined Q8 ONNX)

This repository provides an optimized, high-fidelity INT8 quantized ONNX checkpoint for the Thai Kokoro TTS model (FastThaiG2P lineage).

Optimized and packaged for Text To Speech on Steam.


Key Highlights

  • 33% Smaller Disk Footprint: Reduced from 310.5 MB (FP32) down to 208.4 MB (INT8).
  • 22.5% Lower RAM Usage: Saves 105.3 MB of system RAM during load and reduces peak working-set memory by 118.1 MB.
  • 100% Timing Parity (0.0 ms Duration Drift): Evaluated across 534 rigorous test cases spanning all 44 Thai consonants, digits (0–9, ๐–๙), minimal tone pairs, decimals, currency formats, multi-lingual English loanwords, and massive 475-token paragraphs. The duration predictor matched FP32 timing with exact 0-frame drift (0.0 ms) across all 534 cases.
  • Tonal Pitch Integrity: Mean fundamental frequency ($F_0$) Pearson correlation $r = 0.9907$ with pitch RMSE $< 4.2\text{ Hz}$ (well below human ear pitch discrimination thresholds of $3\text{--}5\text{ Hz}$).
  • Whisper ASR Verified: 100% transcription parity with baseline FP32 when verified on local Whisper large-v3-turbo.
  • Native CPU Acceleration: Optimized for x86_64 AVX2 / VNNI execution on the standard ONNX Runtime CPU Execution Provider.

Quantization Architecture & Policy

Naive static quantization of Kokoro typically ruins duration modeling and introduces vocoder distortion. This model follows the upstream Kokoro quantization policy:

Component Precision Rationale
Thai BERT Encoder (/bert) FP32 Preserves semantic attention and prevents sequence length $>200$ overflow/NaN crashes.
Duration & Pitch LSTMs FP32 Eliminates duration drift and preserves Thai 5-tone contour trajectories.
Vocoder Upsamplers (/decoder/generator/ups) FP32 Prevents metallic noise and artifacts in HiFi-GAN transposed convolutions.
Text Encoder CNNs & Projections INT8 Quantized weights and activations via symmetric static calibration.
AdaIN Linear Layers INT8 Accelerated style conditioning with low computational overhead.

Benchmark Profile (ONNX Runtime CPU - 4 Threads)

Metric Baseline FP32 Refined Q8 Delta / Improvement
Model File Size 310.5 MB 208.4 MB -102.1 MB (-32.9%)
RAM Footprint (Delta) 466.9 MB 361.6 MB -105.3 MB (-22.5%)
Peak Working Set RAM 742.8 MB 624.7 MB -118.1 MB
Cold TTFA (Load + First Sound) 1,199.0 ms 1,354.8 ms ~1.35s from cold launch
Warm Latency (Very Short: 12 tok) 246.8 ms 232.4 ms +5.9% faster (4.4x real-time)
Warm Latency (Short: 32 tok) 445.3 ms 433.7 ms +2.6% faster (4.3x real-time)
Warm Latency (Medium: 98 tok) 935.0 ms 929.9 ms +0.5% faster (4.2x real-time)
Warm Latency (Long: 268 tok) 2,165.4 ms 2,138.2 ms +1.3% faster (3.9x real-time)
Warm Latency (Very Long: 475 tok) 3,417.5 ms 3,384.8 ms +1.0% faster (3.8x real-time)
Duration Drift vs FP32 0.0 ms 0.0 ms (Exact 0 frames) 100% Timing Match
Log-Spectral Distance (LSD) 0.0 dB 5.0 dB Pristine audio fidelity

File Manifest & Integrity

File Size SHA-256 Checksum
kokoro_thai_q8.onnx 208.4 MB D5372F00192E42A334A53FE91EC71E42F11C7FDB6D6667578073698A09235CA1
config.json 2.5 KB EEAC00CF7E2452296DC26998AE71CEBDBF5BF532F3402C93A7F13ACD9847F10E
thai_som.npy 522.4 KB 0E4B0AAAE4BECE614745E5E94849301046A0BFA3F39B6F3EBC152A7DCE598C7F

Usage Example (Python ONNX Runtime)

import onnxruntime as ort
import numpy as np
import soundfile as sf

# Load session on CPU with 4 threads
opts = ort.SessionOptions()
opts.intra_op_num_threads = 4
session = ort.InferenceSession("kokoro_thai_q8.onnx", opts, providers=["CPUExecutionProvider"])

# Load voice style tensor (shape: [1, 256])
style = np.load("thai_som.npy")[0].astype(np.float32)

# Token IDs from FastThaiG2P frontend with BOS (0) and EOS (0)
# Example token sequence for 'ใช่ครับ'
input_ids = np.array([[0, 61, 65, 140, 172, 55, 43, 62, 173, 50, 43, 173, 0]], dtype=np.int64)

# Run inference
outputs = session.run(None, {
    "input_ids": input_ids,
    "ref_s": style,
    "speed": np.array([1.0], dtype=np.float32)
})

audio = outputs[0].squeeze()
sf.write("output.wav", audio, 24000)

Provenance & Attribution

  • Application: Text To Speech on Steam
  • Base Acoustic Model: Derived from FastThaiG2P v0.3.0 (Apache-2.0, AWS Labs).
  • Voice Pack (thai_som.npy): Derived from Som TTS (CC BY).
  • Pronunciation Lexicon & IPA Data: Merged from Kaikki.org / Wiktionary (CC BY-SA 3.0 / GFDL) and validated phonetic dictionaries.
  • License: Apache-2.0.
Downloads last month
85
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support