Add files using upload-large-folder tool

README.md

@@ -0,0 +1,138 @@
+---
+library_name: mlx
+pipeline_tag: text-to-speech
+base_model: stepfun-ai/Step-Audio-EditX
+base_model_relation: quantized
+license: apache-2.0
+language:
+- en
+- zh
+- ja
+- ko
+tags:
+- mlx
+- tts
+- speech
+- voice-cloning
+- audio-editing
+- step-audio
+- step-audio-editx
+- stepfun
+- quantized
+- int8
+- apple-silicon
+- bundled-components
+---
+
+# Step-Audio-EditX — MLX 8-bit
+
+This repository contains a self-contained pure-MLX int8 conversion of
+Step-Audio-EditX for local voice cloning and expressive audio editing on
+Apple Silicon. All pipeline components are stored as `.safetensors` — no
+PyTorch, ONNX, or NumPy files are required at inference time.
+
+## Model Details
+
+- Developed by: AppAutomaton
+- Upstream model: [`stepfun-ai/Step-Audio-EditX`](https://huggingface.co/stepfun-ai/Step-Audio-EditX)
+- Task: zero-shot voice cloning, expressive audio editing
+- Runtime: MLX on Apple Silicon
+- Precision: int8 for Step1 LM, Flow model, and VQ02 tokenizer; bf16 for the rest
+- Total size: ~4.1 GB (down from ~7.7 GB upstream)
+
+## Bundle Contents
+
+This bundle is self-contained — all weights are packaged in one repository.
+
+| File | Component | Format | Size |
+| --- | --- | --- | --- |
+| `model.safetensors` | Step1 LM (3.5B params) | int8 | 3.5 GB |
+| `flow-model.safetensors` | Flow model (DiT + conformer) | int8 | 181 MB |
+| `vq02.safetensors` | VQ02 audio tokenizer | int8 | 162 MB |
+| `vq06.safetensors` | VQ06 audio tokenizer | bf16 | 249 MB |
+| `hift.safetensors` | HiFT vocoder | bf16 | 40 MB |
+| `campplus.safetensors` | CampPlus speaker embedding | bf16 | 13 MB |
+| `flow-conditioner.safetensors` | Flow conditioner | bf16 | 2.5 MB |
+| `config.json` | Step1 LM config + quantization | JSON | — |
+| `flow-model-config.json` | Flow model config | JSON | — |
+| `vq02-config.json`, `vq06-config.json` | Tokenizer configs | JSON | — |
+| `hift-config.json`, `campplus-config.json`, `flow-conditioner-config.json` | Component configs | JSON | — |
+| `tokenizer.json`, `tokenizer.model`, `tokenizer_config.json` | Step1 tokenizer | JSON | — |
+
+## How to Get Started
+
+Install [`mlx-speech`](https://github.com/appautomaton/mlx-speech) and clone
+the repo (the Step Audio CLI entry point is not yet in the unified public API).
+
+```bash
+pip install mlx-speech
+git clone https://github.com/appautomaton/mlx-speech.git
+cd mlx-speech
+```
+
+Download the bundle with `huggingface-cli`:
+
+```bash
+hf download appautomaton/step-audio-editx-8bit-mlx \
+  --local-dir models/stepfun/step_audio_editx/mlx-int8
+```
+
+**Voice cloning:**
+
+```bash
+python scripts/generate_step_audio_editx.py \
+  --model-dir models/stepfun/step_audio_editx/mlx-int8 \
+  --prefer-mlx-int8 \
+  --prompt-audio reference.wav \
+  --prompt-text "Transcript of reference audio." \
+  -o cloned.wav \
+  clone --target-text "New speech in the cloned voice."
+```
+
+**Audio editing (change emotion):**
+
+```bash
+python scripts/generate_step_audio_editx.py \
+  --model-dir models/stepfun/step_audio_editx/mlx-int8 \
+  --prefer-mlx-int8 \
+  --prompt-audio input.wav \
+  --prompt-text "Transcript of input audio." \
+  -o happy.wav \
+  edit --edit-type emotion --edit-info happy
+```
+
+## Supported Edit Types
+
+| Edit type | Description | `--edit-info` examples |
+| --- | --- | --- |
+| `emotion` | Change the emotion of speech | `happy`, `sad`, `angry`, `surprised` |
+| `style` | Change speaking style | `whispering`, `broadcasting`, `formal` |
+| `speed` | Change speaking speed | `fast`, `slow` |
+| `denoise` | Remove noise from audio | not used |
+| `vad` | Remove silences from audio | not used |
+| `paralinguistic` | Add non-verbal sounds | requires `--target-text` |
+
+## Architecture
+
+Five-stage pipeline, all running pure MLX with bf16 activations:
+
+1. **Step1 LM** (3.5B params, int8) — autoregressive dual-codebook token generation
+2. **CampPlus** (bf16) — speaker embedding extraction from reference audio
+3. **Flow conditioner** (bf16) — conditions generation on speaker embedding
+4. **Flow model** (int8) — flow-matching mel spectrogram generation
+5. **HiFT vocoder** (bf16) — mel spectrogram to waveform
+
+The VQ02 and VQ06 tokenizers encode reference audio into dual codebook tokens
+consumed by Step1.
+
+## Performance
+
+On Apple Silicon with int8 weights and bf16 activations, real-time factor
+(RTF) is approximately 1.46x for voice cloning — faster than real-time.
+
+## Links
+
+- Source code: [`mlx-speech`](https://github.com/appautomaton/mlx-speech)
+- Upstream model: [`stepfun-ai/Step-Audio-EditX`](https://huggingface.co/stepfun-ai/Step-Audio-EditX)
+- Technical report: [arXiv:2511.03601](https://arxiv.org/abs/2511.03601)
+- More examples: [AppAutomaton](https://github.com/appautomaton)

campplus-config.json

@@ -0,0 +1,15 @@
+{
+  "feat_dim": 80,
+  "embedding_size": 192,
+  "growth_rate": 32,
+  "init_channels": 128,
+  "block_layers": [
+    12,
+    24,
+    16
+  ],
+  "segment_pool_size": 100,
+  "sample_rate": 16000,
+  "num_mel_bins": 80,
+  "output_level": "segment"
+}

campplus.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:40d2618b188444a982932ff9bfa7f79a1089d38ee58a3a84f592156ad35a0421
+size 13881588

config.json

@@ -0,0 +1,29 @@
+{
+  "hidden_size": 3072,
+  "intermediate_size": 8192,
+  "num_attention_heads": 48,
+  "num_attention_groups": 4,
+  "num_hidden_layers": 32,
+  "vocab_size": 74752,
+  "rms_norm_eps": 1e-05,
+  "bos_token_id": 1,
+  "pad_token_id": 0,
+  "eos_token_id": 3,
+  "tie_word_embeddings": false,
+  "use_cache": true,
+  "max_seq_len": 32768,
+  "model_type": "step1",
+  "architectures": [
+    "Step1ForCausalLM"
+  ],
+  "torch_dtype": "bfloat16",
+  "auto_map": {
+    "AutoConfig": "configuration_step1.Step1Config",
+    "AutoModelForCausalLM": "modeling_step1.Step1ForCausalLM"
+  },
+  "quantization": {
+    "bits": 8,
+    "group_size": 64,
+    "mode": "affine"
+  }
+}

flow-conditioner-config.json

@@ -0,0 +1,10 @@
+{
+  "vocab_size": 5121,
+  "input_size": 512,
+  "output_size": 80,
+  "spk_embed_dim": 192,
+  "vq02_pad_token": 1024,
+  "vq06_prompt_offset": 1024,
+  "vq06_vocoder_base": 1025,
+  "prompt_mel_upsample": 2
+}

flow-conditioner.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:108792964617513bf2b0d8a47255bc8efa7e63acef1b96b866b3d97c0ea8f6a7
+size 2653151

flow-model-config.json

@@ -0,0 +1,28 @@
+{
+  "input_size": 512,
+  "output_size": 80,
+  "spk_embed_dim": 192,
+  "vocab_size": 5121,
+  "encoder_output_size": 512,
+  "pre_lookahead_len": 3,
+  "num_blocks": 6,
+  "num_up_blocks": 4,
+  "up_stride": 2,
+  "up_scale_factor": 2.0,
+  "attention_heads": 8,
+  "linear_units": 2048,
+  "key_bias": true,
+  "estimator_in_channels": 320,
+  "estimator_out_channels": 80,
+  "estimator_hidden_size": 512,
+  "estimator_depth": 16,
+  "estimator_num_heads": 8,
+  "estimator_head_dim": 64,
+  "estimator_mlp_ratio": 4.0,
+  "inference_cfg_rate": 0.7,
+  "quantization": {
+    "bits": 8,
+    "group_size": 64,
+    "mode": "affine"
+  }
+}

flow-model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:40fb429efee588d5c9d124eab6e69517aa80549b04f2710d4495c8f49c046d1a
+size 190272187

hift-config.json

@@ -0,0 +1,72 @@
+{
+  "in_channels": 80,
+  "base_channels": 512,
+  "nb_harmonics": 8,
+  "sampling_rate": 24000,
+  "nsf_alpha": 0.1,
+  "nsf_sigma": 0.003,
+  "nsf_voiced_threshold": 10.0,
+  "upsample_rates": [
+    8,
+    5,
+    3
+  ],
+  "upsample_kernel_sizes": [
+    16,
+    11,
+    7
+  ],
+  "istft_n_fft": 16,
+  "istft_hop_len": 4,
+  "resblock_kernel_sizes": [
+    3,
+    7,
+    11
+  ],
+  "resblock_dilation_sizes": [
+    [
+      1,
+      3,
+      5
+    ],
+    [
+      1,
+      3,
+      5
+    ],
+    [
+      1,
+      3,
+      5
+    ]
+  ],
+  "source_resblock_kernel_sizes": [
+    7,
+    7,
+    11
+  ],
+  "source_resblock_dilation_sizes": [
+    [
+      1,
+      3,
+      5
+    ],
+    [
+      1,
+      3,
+      5
+    ],
+    [
+      1,
+      3,
+      5
+    ]
+  ],
+  "lrelu_slope": 0.1,
+  "audio_limit": 0.99,
+  "f0_predictor": {
+    "num_class": 1,
+    "in_channels": 80,
+    "cond_channels": 512
+  }
+}

hift.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:28a40adb4e2a8bc401b9a12cca2e9219e486e0601ad57b32584a9f3e7e1e41fd
+size 41637433

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7b7fb89f4e58b8a91aa63c237cb5331777c9b7342c34c2e2b1e88952b908e8e1
+size 3750584760

tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:25e122d9205d035033a9994c4d46a6a1b467a938654e4178fc0e5f4f5d610674
+size 1264044

tokenizer_config.json

@@ -0,0 +1,15 @@
+{
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "legacy": false,
+  "model_max_length": 65536,
+  "pad_token": "<unk>",
+  "padding_side": "left",
+  "sp_model_kwargs": {},
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false,
+  "chat_template": "{% if messages[0]['role'] == 'system' %}{{ '<s>' }}{% endif %}{% for message in messages %}{% if message['role'] == 'user' %}{% set role = 'human' %}{% else %}{% set role = message['role'] %}{% endif %}{{ '<|BOT|> ' + role + '\\n' }}{{ message['content'] }}{% if not loop.last or message['role'] != 'assistant' %}{{ '<|EOT|>' }}{% endif %}{% endfor %}{% if add_generation_prompt %}{{ '<|BOT|> assistant\\n' }}{% endif %}"
+}
+

vq02-config.json

@@ -0,0 +1,39 @@
+{
+  "model_name": "ParaformerStreaming",
+  "frontend": {
+    "sample_rate": 16000,
+    "window_type": "hamming",
+    "n_mels": 80,
+    "frame_length_ms": 25.0,
+    "frame_shift_ms": 10.0,
+    "lfr_m": 7,
+    "lfr_n": 6,
+    "dither": 1.0,
+    "snip_edges": true,
+    "remove_dc_offset": true,
+    "preemphasis_coefficient": 0.97,
+    "round_to_power_of_two": true,
+    "low_freq": 20.0,
+    "high_freq": 0.0,
+    "use_power": true,
+    "use_log_fbank": true,
+    "use_energy": false
+  },
+  "encoder": {
+    "input_size": 560,
+    "output_size": 512,
+    "attention_heads": 4,
+    "linear_units": 2048,
+    "num_blocks": 50,
+    "normalize_before": true,
+    "kernel_size": 11,
+    "sanm_shift": 0,
+    "input_layer": "pe_online",
+    "selfattention_layer_type": "sanm"
+  },
+  "quantization": {
+    "bits": 8,
+    "group_size": 64,
+    "mode": "affine"
+  }
+}

vq02.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4caa0d4ef375727e07da750af2ee4c1d9becaf7b9f0018d358db700681593798
+size 169352242

vq06-config.json

@@ -0,0 +1,16 @@
+{
+  "num_mels": 128,
+  "hidden_size": 1280,
+  "num_heads": 20,
+  "num_layers": 6,
+  "max_positions": 1500,
+  "codebook_size": 4096,
+  "conv1_kernel_size": 3,
+  "conv1_stride": 2,
+  "conv1_padding": 1,
+  "conv2_kernel_size": 3,
+  "conv2_stride": 2,
+  "conv2_padding": 1,
+  "layer_norm_eps": 1e-05,
+  "l2_norm_eps": 9.999999960041972e-13
+}

vq06.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3b10c2501908c26ccf14c93995d3110801c5a86221eb919e3481d812a4cb9c86
+size 261268214