Instructions to use drowzeys/keys-GLM-5.3-EXL3-2.75BPW-Abliterated with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use drowzeys/keys-GLM-5.3-EXL3-2.75BPW-Abliterated with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="drowzeys/keys-GLM-5.3-EXL3-2.75BPW-Abliterated")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("drowzeys/keys-GLM-5.3-EXL3-2.75BPW-Abliterated")
model = AutoModelForCausalLM.from_pretrained("drowzeys/keys-GLM-5.3-EXL3-2.75BPW-Abliterated", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use drowzeys/keys-GLM-5.3-EXL3-2.75BPW-Abliterated with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "drowzeys/keys-GLM-5.3-EXL3-2.75BPW-Abliterated"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "drowzeys/keys-GLM-5.3-EXL3-2.75BPW-Abliterated",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/drowzeys/keys-GLM-5.3-EXL3-2.75BPW-Abliterated

SGLang

How to use drowzeys/keys-GLM-5.3-EXL3-2.75BPW-Abliterated with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "drowzeys/keys-GLM-5.3-EXL3-2.75BPW-Abliterated" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "drowzeys/keys-GLM-5.3-EXL3-2.75BPW-Abliterated",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "drowzeys/keys-GLM-5.3-EXL3-2.75BPW-Abliterated" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "drowzeys/keys-GLM-5.3-EXL3-2.75BPW-Abliterated",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use drowzeys/keys-GLM-5.3-EXL3-2.75BPW-Abliterated with Docker Model Runner:
```
docker model run hf.co/drowzeys/keys-GLM-5.3-EXL3-2.75BPW-Abliterated
```

Acknowledge the Responsible Use Agreement to access this repository

Access is granted automatically after you agree to the terms below and submit the form.

Responsible Use Agreement

This model has had safety refusals removed. That makes it useful for red-teaming, security research, evaluation, and unfiltered assistant tasks — and also removes guardrails a user must therefore supply themselves.

Prohibited uses (you must agree before access is granted):

Anything involving the sexual exploitation or endangerment of minors.
You must be of age 18 years or older to use and download this model.
You agree any information generated that can cause harm in terms of generating recipe, knowledge to make any materials/substances is your own input and responsibility. You will be accountable for any harm/damage caused by your action/input.
Content promoting self-harm or suicide.
Generation of material that is illegal in your jurisdiction, or that targets real individuals for harassment, doxxing, or fraud.
Any use prohibited by the upstream Z.AI / GLM MIT license.

You are responsible for adding appropriate safety filtering, human review, and access controls for your deployment. The weights are provided as-is, with no warranty. The license is inherited from the upstream Z.AI GLM-5.3 MIT license — review and comply with it before use or redistribution.

keys-GLM-5.3-EXL3-2.75BPW-Abliterated

Abliterated build of drowzeys/keys-GLM-5.3-EXL3-2.75BPW — same mixed-K EXL3 2.75 bpw pack, same four-Spark TP4 serving shape, with the Blackfrost derisk edit baked into the non-expert EXL3 K5 tensors.

See RESPONSIBLE_USE.md and the gate form above. Access is gated with automatic approval after you agree.


HF	https://huggingface.co/drowzeys/keys-GLM-5.3-EXL3-2.75BPW-Abliterated
Parent (stock 2.75 bpw)	`drowzeys/keys-GLM-5.3-EXL3-2.75BPW`
Derisk donor	`drowzeys/keys-GLM-5.3-EXL3-Abliterated` (Blackfrost α=3.0 on BF16 `o_proj` / residual `down_proj`)
Serve recipe	drowzeys/keys-GLM-5.3-EXL3-2.75BPW-MixedK-Ablit-MLX-Serve-4-DGXSparks
Upstream	`zai-org/GLM-5.3`
Ablit	L2–49 `self_attn.o_proj` + L2 dense `down_proj` + L3–49 shared `down_proj`, re-quantized EXL3 K5 `mul1` (96 matrices / 384 tensors)
Anchors	L0–1 and L50–78 (including MTP), routed experts, embed, `lm_head` stay parent stock
Gate (thinking off)	refusal32 31/32 (item 4 still refuses) · cyber 22/22 · 0 garble

What was edited

The 2.75 bpw checkpoint stores attention and shared-expert linears as EXL3 K5 mul1, so the donor's native F16 tensors cannot be dropped in. The same Blackfrost derisk tensors from the 3 bpw ablit pack were re-encoded to K5 (data-free fallback, proxy ~7–9e-4) and spliced into shards model-00002 and model-00003.

Routed experts (mixed 2/3/4-bit, mean 2.75), MTP, embeddings and the head are bit-identical to the parent.

ABLIT.json in this repo records the splice.

Run it (four DGX Sparks)

Use the launcher (one-shot.sh) and prebuilt image of the serve recipe.

hf download drowzeys/keys-GLM-5.3-EXL3-2.75BPW-Abliterated --local-dir /models/GLM-5.3-EXL3-2.75BPW-Abliterated
export NODES="spark1 spark2 spark3 spark4" MODEL=/models/GLM-5.3-EXL3-2.75BPW-Abliterated
./one-shot.sh check && ./one-shot.sh up && ./one-shot.sh wait

OpenAI-compatible API on rank 0; the port and model name are in the serve recipe. Speed matches the parent (prose ~42 tok/s / code ~38 tok/s on the published image, thinking on).

Faster drafts for these weights (2026-10-07)

drowzeys/keys-GLM-5.3-speculator.dspark-ft2 is a DSpark drafter fine-tuned on this checkpoint's own thinking-on outputs (GLM-5.3 license). With image 2026-10-07 and DSPARK=/models/keys-GLM-5.3-speculator.dspark-ft2 ./one-shot.sh up: prose 33.8 / code 39.0 tok/s at 32K with thinking on, against 32.2 / 34.8 with the built-in MTP head (+5 % / +12 %); 1M context validated.

Layout

Part	Format	Bits
Routed experts (layers 3–77, 256 each)	EXL3 trellis, `mul1`, per-expert width	mean 2.75
MTP layer (78) experts	EXL3 `mul1`	8
Attention, shared experts, dense MLPs 0–2	EXL3 `mul1`	5 (L2–49 `o_proj` / residual `down_proj` are the derisked K5)
`kv_b`, indexer `wk` / `weights_proj`, router, norms, embeddings, `lm_head`	BF16	16

81 safetensors shards, ~258 GB. kv_a_proj_with_mqa is stored 640 wide (576 real + zero pad).

6-bit output head (optional, added 2026-10-12)

lm_head-6bit/lm_head-6bit.safetensors (714 MB) is lm_head.weight quantized to EXL3 at 6.0 bits with exllamav3 (tensors lm_head.trellis, lm_head.suh, lm_head.svh, lm_head.mul1). This repository's lm_head.weight is byte-identical to the unmodified pack's, so it is the same file as in drowzeys/keys-GLM-5.3-EXL3-2.75BPW. The checkpoint's own lm_head.weight stays bf16 and nothing else changed.

The recipe's 2026-10-12 image reads the head from this file when it is in the checkpoint folder (hf download puts it there), and from the bf16 tensor when it is not or with HEAD_Q=off. Measured on the unmodified pack, as served on four DGX Sparks, against the BF16 checkpoint over 137,490 positions: KL 0.175612 with the bf16 head and 0.175724 with this one, top-1 agreement 89.64% and 89.60%; a head launch takes 0.79 ms against 2.16 ms.

Credits

Z.ai — GLM-5.3.
Blackfrost AI (Blackfrost-Research) — BF16 derisk projections (GLM-5.3-DERISKED-BF16) adapted here via the Keys 3 bpw EXL3 ablit pack.
The serving stack's third-party software and licences are listed in the serve recipe's NOTICE; its inference engine's project is not affiliated with this model and does not endorse or test it. MiaAI-Lab, Jay Leaton, BertholomusAI, turboderp / ExLlamaV3, vcruz305, b12x, NVIDIA — serving stack as on the parent card.

Use is subject to the GLM-5.3 license of the base model and the Responsible Use terms above.

Downloads last month: 237

Safetensors

Model size

138B params

Tensor type

F32

BF16

F16

I16

Model tree for drowzeys/keys-GLM-5.3-EXL3-2.75BPW-Abliterated

Base model

zai-org/GLM-5.3

Quantized

drowzeys/keys-GLM-5.3-EXL3-2.75BPW

Finetuned

(1)

this model