Download llama_modeling/config.py from Blackroot/Differential-Microllama-2: direct link, hf CLI and curl.
- Browser
- Download file 419 Bytes
-
https://huggingface.co/Blackroot/Differential-Microllama-2/resolve/main/llama_modeling/config.py
- Command line
-
hf download hf://Blackroot/Differential-Microllama-2/llama_modeling/config.py
-
curl -L -o config.py https://huggingface.co/Blackroot/Differential-Microllama-2/resolve/main/llama_modeling/config.py
419 Bytes
| from dataclasses import dataclass | |
| class LlamaConfig: | |
| hidden_size: int = 576 | |
| num_attention_heads: int = 16 | |
| num_key_value_heads: int = 4 | |
| num_hidden_layers: int = 30 | |
| intermediate_size: int = 1536 | |
| hidden_act: str = "silu" | |
| rms_norm_eps: float = 1e-5 | |
| vocab_size: int = 49152 | |
| max_position_embeddings: int = 8192 | |
| rope_theta: int = 100000 | |
| tie_word_embeddings: bool = False |