SkyIsNotGreen commited on
Commit
2d78a22
·
verified ·
1 Parent(s): e64a754

model card: add seven figures and the data/plot references

Browse files
.gitattributes CHANGED
@@ -1 +1,6 @@
1
  *.gguf filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
1
  *.gguf filter=lfs diff=lfs merge=lfs -text
2
+ figures/flashnext-664-truncation.png filter=lfs diff=lfs merge=lfs -text
3
+ figures/flashnext-at-a-glance.png filter=lfs diff=lfs merge=lfs -text
4
+ figures/flashnext-kld-gate.png filter=lfs diff=lfs merge=lfs -text
5
+ figures/flashnext-q4x160-trade.png filter=lfs diff=lfs merge=lfs -text
6
+ figures/flashnext-quality-ab.png filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -43,6 +43,13 @@ file at 60.17 GB and the F16-body file at 65.80 GB.
43
  The name comes from grafting. A scion is the shoot grafted onto a rootstock, and
44
  here the trained corrections are grafted onto a 1.75 bpw ternary expert body.
45
 
 
 
 
 
 
 
 
46
  ## Performance
47
 
48
  Measured with **Strata**, this project's own inference engine (a separate runtime from the
@@ -59,6 +66,12 @@ the expert caches across both cards:
59
  - Protocol: four requests of one fixed prompt, run 1 warms and runs 2-4 report, `temperature=0`,
60
  timings from the engine's own record. The embedded corrections are applied; the file is unmodified.
61
 
 
 
 
 
 
 
62
  ## Highlights
63
 
64
  - **One 57.62 GB file at 2.60 bpw, the smallest of three shipped containers.** The official FP8 release of the same model is 185.6 GB; the published community quants are 102 GB (ggml-org IQ4_NL), 92 GB (the only other ternary build, Mooney PQ2_0), 83.6 GB (GSQ-RCO IQ3_S), 75.8 GB (IQ3_XXS), 68.0 GB (IQ2_XS) and 66.4 GB (GSQ-RCO Q2_0). This is the smallest build of this model on the Hub we can see, **pruned or not**: the next smallest file is ISTA's **expert-pruned** coder IQ1_M at 58.4 GB, and the other two Scion files are 60.17 GB (Q4_0 table) and 65.80 GB (F16 body).
@@ -74,6 +87,9 @@ the expert caches across both cards:
74
  - **[GitHub `sky-is-green/scion`](https://github.com/sky-is-green/scion)**: the source of truth for this work — the harness that produced the file, the port decisions, the measured traps and the negative register.
75
  - **Runtime**: [`sky-is-green/prism-ml-llama.cpp`](https://github.com/sky-is-green/prism-ml-llama.cpp/tree/qwen4exp-proto), branch `qwen4exp-proto`, a fork of Prism ML's llama.cpp with the PQ2_0/PTQ1_0 expert containers and the Q4X160/Q4_0 PLE tables, the qwen4exp architecture, the `ffn_moe_out` virtual target, embedded-adapter support and PLE table handling.
76
  - **[Discussions](https://huggingface.co/SkyIsNotGreen/Scion-FlashNext-176B-A6B/discussions)**: questions, test reports and failures are all welcome.
 
 
 
77
 
78
  ## Model Overview
79
 
@@ -173,6 +189,13 @@ The published artifact: `Scion-FlashNext-176B-A6B-Q6_K-Q4X160.gguf`,
173
  (experts untouched; the table stream in it is the prebuilt
174
  `work/tmp-export/q4x160-table.bin`, sha256 `915bae37…3015`).
175
 
 
 
 
 
 
 
 
176
  ## Best Practices
177
 
178
  ### Generation Parameters
@@ -290,6 +313,12 @@ split of the mean: **support 0.5387 (92.9%)**, marginal 0.0270 (4.7%), tail
290
  teacher mass), so the support fit is what was trained; the tail is nearly
291
  inert by design.
292
 
 
 
 
 
 
 
293
  ### Community benchmark: released Q6K vs ISTA GSQ-RCO Q2_0
294
 
295
  Measured 2026-10-06 in one A100-SXM4-80GB session (US-WA-1) with
@@ -316,6 +345,14 @@ LiveCodeBench v6 81.14; task average 89.07, zero-shot average 78.00) is from
316
  their model card, uses a different suite, and was **not measured here** — it is
317
  context, not an A/B result.
318
 
 
 
 
 
 
 
 
 
319
  ### Decision-arm benchmark: 664-task reasoning/code suite (2026-10-09)
320
 
321
  The project's community arms (`experiments/cascade/community_eval.py`) run on
@@ -373,6 +410,17 @@ long thinking budgets). The judge heads are unaffected (prompt embeddings, no
373
  generation), and the train-code think-harder drop (53.9%) is the same
374
  truncation effect at its worst.
375
 
 
 
 
 
 
 
 
 
 
 
 
376
  Two fork-side fixes were needed to run the suite end-to-end: the server's
377
  prompt cache disabled (`--cache-ram 0`, its eviction path faults with the
378
  hybrid GDN KV) and the embedding read width corrected. The measurements ran on
@@ -391,6 +439,12 @@ Both with 0 harness errors. The Q6_K body requantises only the body (experts,
391
  table and corrections are byte-identical to the F16-body file), and the two
392
  scores are within the suite's noise band.
393
 
 
 
 
 
 
 
394
  ### Generation proof (temperature 0, 32 tokens)
395
 
396
  | prompt | output |
 
43
  The name comes from grafting. A scion is the shoot grafted onto a rootstock, and
44
  here the trained corrections are grafted onto a 1.75 bpw ternary expert body.
45
 
46
+ ![Where the 57.62 GB goes and how it compares with the published builds](./figures/flashnext-at-a-glance.png)
47
+
48
+ *The default file is 57.62 GB: 26.42 GB of PTQ1_0 experts, 26.24 GB of Q4X160
49
+ n-gram table, and 4.95 GB of body, corrections and headers. It is 3.2x smaller
50
+ than the 185.6 GB FP8 release and smaller than every published community quant
51
+ below, including the expert-pruned coder IQ1_M.*
52
+
53
  ## Performance
54
 
55
  Measured with **Strata**, this project's own inference engine (a separate runtime from the
 
66
  - Protocol: four requests of one fixed prompt, run 1 warms and runs 2-4 report, `temperature=0`,
67
  timings from the engine's own record. The embedded corrections are applied; the file is unmodified.
68
 
69
+ ![Measured generation performance on the local two-card box](./figures/flashnext-performance.png)
70
+
71
+ *Measured on the local two-card box: 80.5-80.8 tok/s generation, 12.4 ms/token,
72
+ 2.3-2.4 tokens per pipeline window. This is the 60.17 GB Q4_0-table file; the
73
+ 57.62 GB default is not measured on this box yet.*
74
+
75
  ## Highlights
76
 
77
  - **One 57.62 GB file at 2.60 bpw, the smallest of three shipped containers.** The official FP8 release of the same model is 185.6 GB; the published community quants are 102 GB (ggml-org IQ4_NL), 92 GB (the only other ternary build, Mooney PQ2_0), 83.6 GB (GSQ-RCO IQ3_S), 75.8 GB (IQ3_XXS), 68.0 GB (IQ2_XS) and 66.4 GB (GSQ-RCO Q2_0). This is the smallest build of this model on the Hub we can see, **pruned or not**: the next smallest file is ISTA's **expert-pruned** coder IQ1_M at 58.4 GB, and the other two Scion files are 60.17 GB (Q4_0 table) and 65.80 GB (F16 body).
 
87
  - **[GitHub `sky-is-green/scion`](https://github.com/sky-is-green/scion)**: the source of truth for this work — the harness that produced the file, the port decisions, the measured traps and the negative register.
88
  - **Runtime**: [`sky-is-green/prism-ml-llama.cpp`](https://github.com/sky-is-green/prism-ml-llama.cpp/tree/qwen4exp-proto), branch `qwen4exp-proto`, a fork of Prism ML's llama.cpp with the PQ2_0/PTQ1_0 expert containers and the Q4X160/Q4_0 PLE tables, the qwen4exp architecture, the `ffn_moe_out` virtual target, embedded-adapter support and PLE table handling.
89
  - **[Discussions](https://huggingface.co/SkyIsNotGreen/Scion-FlashNext-176B-A6B/discussions)**: questions, test reports and failures are all welcome.
90
+ - **Figures**: the plots in this card are generated from
91
+ `moe/results/qwen4exp/flashnext-card-data.json` by
92
+ `moe/plot_flashnext_card.py` in the GitHub repo.
93
 
94
  ## Model Overview
95
 
 
189
  (experts untouched; the table stream in it is the prebuilt
190
  `work/tmp-export/q4x160-table.bin`, sha256 `915bae37…3015`).
191
 
192
+ ![PLE table container trade: Q4X160 vs Q3X32 vs Q4_0](./figures/flashnext-q4x160-trade.png)
193
+
194
+ *Q4X160 cuts the table from 28.8 GB to 26.24 GB and the file from 60.17 GB to
195
+ 57.62 GB, at +0.41% PPL, -0.0001 KLD, 0.00 HellaSwag and +0.25 Winogrande.
196
+ Q3X32 is 6.41 GB smaller but costs +1.30% PPL and +0.0108 KLD, so it stays a
197
+ fallback and is not shipped.*
198
+
199
  ## Best Practices
200
 
201
  ### Generation Parameters
 
313
  teacher mass), so the support fit is what was trained; the tail is nearly
314
  inert by design.
315
 
316
+ ![48-layer KLD gate: step-3000 vs step-4000](./figures/flashnext-kld-gate.png)
317
+
318
+ *The released step-3000 has mean KLD 0.5800; the step-4000 extension is
319
+ flat/worse at 0.5880 and is not shipped. 92.9% of the mean is the top-512
320
+ support fit, 4.7% marginal and 2.5% tail.*
321
+
322
  ### Community benchmark: released Q6K vs ISTA GSQ-RCO Q2_0
323
 
324
  Measured 2026-10-06 in one A100-SXM4-80GB session (US-WA-1) with
 
345
  their model card, uses a different suite, and was **not measured here** — it is
346
  context, not an A/B result.
347
 
348
+ ![Scion vs ISTA GSQ-RCO Q2_0 on the same 400 tasks](./figures/flashnext-quality-ab.png)
349
+
350
+ *Same harness, same 400 tasks. Blue = Scion, grey = ISTA; whiskers are the
351
+ published CIs (PPL ±0.032/±0.033, Winogrande ±2.09/±2.18; HellaSwag has no
352
+ published CI). Scion trails ISTA on PPL by 4.2%, ties on HellaSwag (paired
353
+ p = 0.83), and leads on Winogrande by 2.75 points (paired p = 0.22, not
354
+ conclusive). The size difference is 6.2 GB, or 8.8 GB with the Q4X160 table.*
355
+
356
  ### Decision-arm benchmark: 664-task reasoning/code suite (2026-10-09)
357
 
358
  The project's community arms (`experiments/cascade/community_eval.py`) run on
 
410
  generation), and the train-code think-harder drop (53.9%) is the same
411
  truncation effect at its worst.
412
 
413
+ ![664-task accuracy and truncation at the protocol caps](./figures/flashnext-664-truncation.png)
414
+
415
+ *The headline is a floor, not a capability number. "Completed" means the record
416
+ finished inside the fixed token cap; it is not the model choosing to answer. On
417
+ think-harder, 175 of the 178 cap-hit records emitted no final answer because the
418
+ 2048-token cap stopped the thinking trace before the answer; the gen arm hit the
419
+ 700-token cap on 16% of records but left no record without an answer. Scoring
420
+ only records that completed inside the cap gives 82.6% / 91.8%; including
421
+ cap-hit records gives the 69.4% / 67.6% floor. A fair capability number needs a
422
+ larger thinking budget.*
423
+
424
  Two fork-side fixes were needed to run the suite end-to-end: the server's
425
  prompt cache disabled (`--cache-ram 0`, its eviction path faults with the
426
  hybrid GDN KV) and the embedding read width corrected. The measurements ran on
 
439
  table and corrections are byte-identical to the F16-body file), and the two
440
  scores are within the suite's noise band.
441
 
442
+ ![Capability screen: Q6_K body vs F16 body](./figures/flashnext-capability.png)
443
+
444
+ *Q6_K scores 15/20 and F16 14/20; code and logic are 1/3 each in both. The
445
+ Q6_K body requantises only the body; the experts, table and corrections are
446
+ byte-identical to the F16-body file.*
447
+
448
  ### Generation proof (temperature 0, 32 tokens)
449
 
450
  | prompt | output |
figures/flashnext-664-truncation.png ADDED

Git LFS Details

  • SHA256: 20625b298a17b91f85330b0eab314a2bafd0619142667d8e8774dee66e11d288
  • Pointer size: 131 Bytes
  • Size of remote file: 153 kB
figures/flashnext-at-a-glance.png ADDED

Git LFS Details

  • SHA256: e384c6a7bcd2a7d5c11295634c7661b4e201fa605e7d2dc1158b504d899d4465
  • Pointer size: 131 Bytes
  • Size of remote file: 207 kB
figures/flashnext-capability.png ADDED
figures/flashnext-kld-gate.png ADDED

Git LFS Details

  • SHA256: e4cb988610b94ebb6ed53ff9ed128ba51bf6df4bff95bc2f30dd69604f362911
  • Pointer size: 131 Bytes
  • Size of remote file: 163 kB
figures/flashnext-performance.png ADDED
figures/flashnext-q4x160-trade.png ADDED

Git LFS Details

  • SHA256: 8e541db48836e83840911eddd592554f64626cbe148e4c9c0311298201ce9373
  • Pointer size: 131 Bytes
  • Size of remote file: 171 kB
figures/flashnext-quality-ab.png ADDED

Git LFS Details

  • SHA256: 48cde82bec263f44ab873d32a7f0e5d05b782d8f3e22ac3fa1c0493101758912
  • Pointer size: 131 Bytes
  • Size of remote file: 169 kB