buley commited on
Commit
b3ee817
·
verified ·
1 Parent(s): 4537c40

Update model card with proper metadata and documentation

Browse files
Files changed (1) hide show
  1. README.md +75 -0
README.md ADDED
@@ -0,0 +1,75 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ license: apache-2.0
5
+ library_name: gguf
6
+ tags:
7
+ - gguf
8
+ - vibevoice
9
+ - realtime
10
+ - tts
11
+ - speech
12
+ - tiny
13
+ - affectively
14
+ - edgework
15
+ - aether
16
+ - distributed-inference
17
+ - edge-deployment
18
+ base_model: fixie-ai/ultravox-v0_5-llama-3_2-1b
19
+ base_model_relation: quantized
20
+ pipeline_tag: text-to-speech
21
+ ---
22
+
23
+ # VibeVoice Realtime 0.5B (GGUF, Q4_K_M)
24
+
25
+ > **Production-ready** GGUF quantization of [fixie-ai/ultravox-v0_5-llama-3_2-1b](https://huggingface.co/fixie-ai/ultravox-v0_5-llama-3_2-1b) for distributed speech synthesis — powered by the [Aether](https://github.com/affectively-ai/aether) edge inference runtime.
26
+
27
+ ## Highlights
28
+
29
+ - **0.5B parameters** — Ultra-low-latency voice model. Designed for real-time speech synthesis and voice interaction.
30
+ - **~0.3 GB** Q4_K_M quantized — optimized for distributed edge inference
31
+ - **LLaMA architecture** — proven, stable, well-tested
32
+ - **Aether runtime compatible** — layer-sharded across distributed nodes via [Edgework.ai](https://edgework.ai)
33
+
34
+ ## Model Details
35
+
36
+ | Property | Value |
37
+ |----------|-------|
38
+ | Base model | [fixie-ai/ultravox-v0_5-llama-3_2-1b](https://huggingface.co/fixie-ai/ultravox-v0_5-llama-3_2-1b) |
39
+ | Parameters | 0.5B |
40
+ | Architecture | LLaMA |
41
+ | Quantization | Q4_K_M |
42
+ | Format | GGUF |
43
+ | Size | ~0.3 GB |
44
+ | License | apache-2.0 |
45
+
46
+ ## Usage
47
+
48
+ ### With llama.cpp
49
+
50
+ ```bash
51
+ ./llama-cli -m vibevoice-realtime-0.5b-q4_k_m.gguf -p "Your prompt here" -n 256
52
+ ```
53
+
54
+ ### With Aether (Distributed Inference)
55
+
56
+ This model is deployed across the [Aether](https://github.com/affectively-ai/aether) distributed inference network. Weights are layer-sharded and distributed across multiple edge nodes for parallel inference.
57
+
58
+ ## Deployment Architecture
59
+
60
+ This model runs on the **Aether distributed inference runtime** — our custom engine that shards model layers across multiple nodes for parallel execution:
61
+
62
+ 1. **Coordinator** receives requests and manages token generation
63
+ 2. **Layer nodes** each hold a subset of model layers
64
+ 3. **Hidden states flow** between nodes via gRPC
65
+ 4. **Zero cold start** via warm pool scheduling
66
+
67
+ Deployed via [Edgework.ai](https://edgework.ai) — bringing fast, cheap, and private inference as close to the user as possible.
68
+
69
+ ## About
70
+
71
+ Published by [AFFECTIVELY](https://huggingface.co/affectively-ai) · Managed by [@buley](https://huggingface.co/buley)
72
+
73
+ We quantize and publish **production-ready models** for distributed edge inference via the [Aether](https://github.com/affectively-ai/aether) runtime. Every release is tested for correctness and stability before publication.
74
+
75
+ - [All models](https://huggingface.co/affectively-ai) · [GitHub](https://github.com/affectively-ai) · [Edgework.ai](https://edgework.ai)