gte-modernbert-base β€” GGUF Q8_0

Alibaba-NLP/gte-modernbert-base converted to GGUF at Q8_0 for llama.cpp's llama-server --embeddings, with the ModernBERT architecture recorded correctly.

File gte-modernbert-base-Q8_0.gguf (160 MB)
SHA-256 ddae471b78b3380e3e0db99b14ca588428b6592d3df9f6b3d571359087ae8c1f
Source revision e7f32e3c00f91d699e8c43b53106206bcc72bb22 (model.safetensors sha256 3e85899d…)
Architecture modern-bert
Context 8192 tokens
Pooling CLS (stored in the file, as the model card prescribes)
Dimensions 768
llama-server -m gte-modernbert-base-Q8_0.gguf --embeddings --port 8080
curl -s localhost:8080/v1/embeddings -H 'Content-Type: application/json' -d '{"input":["def f(): pass"]}'

Why this conversion

A GGUF of this model we had downloaded elsewhere was labelled bert with no bert.context_length, and llama.cpp refused it. This one was made with llama.cpp's own convert_hf_to_gguf.py (2026-10-01, --outtype q8_0), which maps the model to llama.cpp's modern-bert architecture; it needs a llama.cpp build that has that architecture.

Verification

Embedded short code and prose inputs with this file on llama-server and with the original model in PyTorch (CLS vector of the last hidden state): cosine similarity 0.9997–0.9998.

The weights and their licence (Apache-2.0) are Alibaba-NLP's.

Downloads last month
40
GGUF
Model size
0.1B params
Architecture
modern-bert
Hardware compatibility
Log In to add your hardware

8-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for ChrisGVE/gte-modernbert-base-Q8_0-GGUF

Quantized
(16)
this model