Hugging Face's logo

librepowerai
/
Granite-4.1-8B-Power

GGUF
q4_k_m
ppc64le
ibm-power
aix
llama.cpp
Model card Files
xet
 Community
How to use from
llama.cpp
Install (macOS, Linux) 
curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf librepowerai/Granite-4.1-8B-Power:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf librepowerai/Granite-4.1-8B-Power:Q4_K_M
Install from WinGet (Windows) 
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf librepowerai/Granite-4.1-8B-Power:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf librepowerai/Granite-4.1-8B-Power:Q4_K_M
Build from source code 
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf librepowerai/Granite-4.1-8B-Power:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama cli -hf librepowerai/Granite-4.1-8B-Power:Q4_K_M
Use Docker 
docker model run hf.co/librepowerai/Granite-4.1-8B-Power:Q4_K_M
Quick Links

Granite-4.1-8B — Q4_K_M for IBM Power (Linux ppc64le + AIX)

Granite-4.1-8B quantized to Q4_K_M with a Q6_K output head for fast CPU inference on IBM Power — POWER9 (VSX) and POWER10/11 (MMA-accelerated) via LibrePower. No GPU required. Size: 5.0G.

Run it

Ubuntu / Debian ppc64le:

curl -fsSL https://linux.librepower.org/install.sh | sudo sh
sudo apt install librepower-llama
wget https://huggingface.co/librepowerai/Granite-4.1-8B-Power/resolve/main/Granite-4.1-8B-Q4_K_M.gguf
lp-llama-completion -m Granite-4.1-8B-Q4_K_M.gguf -p "Hello!" -n 64 -t $(nproc)

IBM AIX 7.3 (big-endian):

dnf install llama-aix
wget https://huggingface.co/librepowerai/Granite-4.1-8B-Power/resolve/main/Granite-4.1-8B-Q4_K_M-be.gguf
lp-llama-completion -m Granite-4.1-8B-Q4_K_M-be.gguf -p "Hello!" -n 64 -t $(nproc)

Files

  • Granite-4.1-8B-Q4_K_M.gguf — little-endian (Ubuntu/Linux ppc64le)
  • Granite-4.1-8B-Q4_K_M-be.gguf — big-endian (IBM AIX)

Good for

IBM 8B dense, signed, ISO-42001: enterprise RAG, tool-calling, structured JSON, code

Credits

Base model by its original authors (Apache-2.0). Quantization & Power packaging: LibrePower.

Downloads last month
54
GGUF
Model size
9B params
Architecture
granite
Hardware compatibility
Log In to add your hardware

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for librepowerai/Granite-4.1-8B-Power

Base model

ibm-granite/granite-4.1-8b
Quantized
(82)
this model