README.md

---
language: en
library_name: mlx
pipeline_tag: image-text-to-text
base_model: thinkingmachines/Inkling-Small
tags:
- mlx
---

# Inkling Small MXFP4

MXFP4 version of [thinkingmachines/Inkling-Small](https://huggingface.co/thinkingmachines/Inkling-Small).

This checkpoint uses less memory than [thinkingmachines/Inkling-Small-NVFP4](https://huggingface.co/thinkingmachines/Inkling-Small-NVFP4),
but quantizes more tensors to MXFP4. The official NVFP4 checkpoint, on the other hand, only applies 4-bit quantization to routed experts,
so quality should be better.

For the best quality, please run the official NVFP4 checkpoint directly with MLX, there's no need to convert:

```bash
mlx_vlm.generate --prompt "who are you?" --model thinkingmachines/Inkling-Small-NVFP4
```

Or, if you want to try the version in this repo (lower memory consumption, faster):

```bash
mlx_vlm.generate --prompt "who are you?" --model mlx-community/Inkling-Small-mxfp4
```

**Note**: please make sure you use the Inkling Small mlx-vlm PR.