microsoft

/

Phi-4-mini-flash-reasoning

like 256

Follow

Microsoft 17.4k

Phi-4-mini-flash-reasoning
isn't readily available in GGUF format because its unique SambaY architecture (a Mamba variant) differs from traditional Transformer models, complicating direct conversion to GGUF, which is optimized for LLama/Transformer structures, though efforts are underway by the community to support its efficient, low-latency, long-context performance on consumer hardware.
Why the Confusion/Difficulty?

New Architecture: Unlike the original Phi-4-mini (which is Transformer-based and easily converts to GGUF), the "flash" version uses a State Space Model (SSM) backbone called SambaY, which has a different computational structure.
GGUF's Focus: GGUF (GPT-Generated Unified Format) was primarily designed to efficiently run Transformer-based models (like Llama, Mistral) on CPUs and GPUs using tools like llama.cpp.
Conversion Challenges: The different architecture means standard conversion scripts (like hf-to-gguf) struggle or fail because they expect Transformer layers, not SambaY's unique self-decoder/cross-decoder setup. 

What's the Goal (and Solution)?

Speed & Context: The Flash model offers much lower latency and better long-context handling due to its architecture, making it great for production.
Community Efforts: Enthusiasts and developers are working on creating specific tools or adapting llama.cpp to support this new architecture for local inference, similar to how the original Phi-4-mini was made accessible. 

In short, it's a format compatibility issue due to a new, efficient underlying model design, not a bug, and people are working on making it work.

Oh I see, thanks man !