librepowerai/Granite-4.1-8B-Power

Granite-4.1-8B — Q4_K_M for IBM Power (Linux ppc64le + AIX)

Granite-4.1-8B quantized to Q4_K_M with a Q6_K output head for fast CPU inference on IBM Power — POWER9 (VSX) and POWER10/11 (MMA-accelerated) via LibrePower. No GPU required. Size: 5.0G.

Run it

Ubuntu / Debian ppc64le:

curl -fsSL https://linux.librepower.org/install.sh | sudo sh
sudo apt install librepower-llama
wget https://huggingface.co/librepowerai/Granite-4.1-8B-Power/resolve/main/Granite-4.1-8B-Q4_K_M.gguf
lp-llama-completion -m Granite-4.1-8B-Q4_K_M.gguf -p "Hello!" -n 64 -t $(nproc)

IBM AIX 7.3 (big-endian):

dnf install llama-aix
wget https://huggingface.co/librepowerai/Granite-4.1-8B-Power/resolve/main/Granite-4.1-8B-Q4_K_M-be.gguf
lp-llama-completion -m Granite-4.1-8B-Q4_K_M-be.gguf -p "Hello!" -n 64 -t $(nproc)

Files

• Granite-4.1-8B-Q4_K_M.gguf — little-endian (Ubuntu/Linux ppc64le)
• Granite-4.1-8B-Q4_K_M-be.gguf — big-endian (IBM AIX)

Good for

IBM 8B dense, signed, ISO-42001: enterprise RAG, tool-calling, structured JSON, code

Credits

Base model by its original authors (Apache-2.0). Quantization & Power packaging: LibrePower.