Sky-Kim/com.sky.sentis.ecapa512

sentis-ecapa512

WeSpeaker ECAPA-TDNN (C=512) speaker embedding (VoxCeleb) converted to Unity Inference Engine (Sentis) FP32.

Files

ecapa512_fp32.sentis   # fbank [1, 300, 80] -> 192-d speaker embedding
EcapaTdnn512.cs   # self-contained Unity Sentis inference

Inference

The model expects 80-bin Kaldi log-mel filterbanks over a fixed 3 s window (300 frames, 25 ms / 10 ms), with the per-utterance mean removed. EcapaTdnn512 computes these features in C# from 16 kHz mono audio, splits longer audio into 3 s windows with a 1.5 s hop (shorter audio is wrap-padded) and averages the L2-normalized window embeddings. Two utterances are compared by cosine similarity (a dot product, since embeddings are normalized); the nested EcapaTdnn512.Voiceprints keeps one averaged prototype per enrolled speaker.

A complete self-contained implementation lives in EcapaTdnn512.cs (Kaldi fbank front-end, windowing, sync + async readback, enrollment/identification). Minimal usage:

var model = new EcapaTdnn512(BackendType.CPU);
model.Load(modelRoot); // folder holding ecapa512_fp32.sentis
model.Warmup();

var voiceprints = new EcapaTdnn512.Voiceprints(threshold: 0.5f);
voiceprints.Enroll("alice", model.Embed(aliceSamples));   // 16 kHz mono float[] in [-1, 1]
voiceprints.Enroll("alice", model.Embed(aliceSamples2));  // 2-5 utterances per speaker work best

var match = voiceprints.Identify(model.Embed(unknownSamples));
// match.Speaker == null when the best cosine is below the threshold (unknown speaker)

Run one embedding per finished utterance (e.g. after a VAD end-of-speech event), not per frame. For a GPU backend use EmbedAsync; a synchronous GPU readback can stall on some mobile GPUs. Thresholds are model-specific: calibrate on your own enrollment / impostor recordings (0.5 is a starting point).

Conversion notes

• Source: voxceleb_ECAPA512.onnx from Wespeaker/wespeaker-voxceleb-ecapa-tdnn512.
• Input fixed to [1, 300, 80] and simplified with onnxsim (ONNX Runtime output unchanged, cosine 1.000000).
• Unity Sentis (CPU) output matches the original model with cosine 1.000000. The C# fbank matches torchaudio.compliance.kaldi.fbank within 2e-4.

WeSpeaker reports VoxCeleb1-O-clean EER 1.069 % for ECAPA_TDNN_GLOB_c512-ASTP-emb192 (cosine scoring, no AS-Norm).

License & attribution

Apache-2.0. Converted from Wespeaker/wespeaker-voxceleb-ecapa-tdnn512 (Apache-2.0) by the WeSpeaker team. Wrapper code: MIT. Paper: ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification.