LiquidAI
/

LFM2-8B-A1B

Text Generation

Mixture of Experts

Model card Files Files and versions

mlabonne commited on Oct 8

Commit

a62c8a1

·

verified ·

1 Parent(s): 19a7cba

fix typo

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -93,7 +93,7 @@ However, we do not recommend using them for tasks that are knowledge-intensive o
 | --------------------- | ----------------------------- |
 | **Total parameters**  | 8.3B                          |
 | **Active parameters** | 1.5B                          |
-| **Layers**            | 24 (10 conv + 6 attn)         |
 | **Context length**    | 32,768 tokens                 |
 | **Vocabulary size**   | 65,536                        |
 | **Training precision**| Mixed BF16/FP8                |

 | --------------------- | ----------------------------- |
 | **Total parameters**  | 8.3B                          |
 | **Active parameters** | 1.5B                          |
+| **Layers**            | 24 (18 conv + 6 attn)         |
 | **Context length**    | 32,768 tokens                 |
 | **Vocabulary size**   | 65,536                        |
 | **Training precision**| Mixed BF16/FP8                |