Nanbeige4.3-33B-oQ4e

This repository holds a complete Apple MLX build of Nanbeige4.3-33B, quantized with oMLX's oQ method at a nominal 4 bits.

Nanbeige4.3-33B is the Nanbeige team's flagship of the 4.3 series: a dense 33B model for agentic coding and office work ("cowork"), with a reasoning mode and tool calling. The team built it as the teacher for smaller models of the series. The source model card has its benchmark results.

Download the whole repository: the Safetensors shards need the included index, model config, tokenizer and chat template. This is not a GGUF or Transformers artifact.

Quick facts

Item	Value
Model type	llama (LlamaForCausalLM), dense
Size	33.1B parameters; 52 layers, hidden size 7,168, 56 attention heads with 8 KV heads
Quantization layout	Affine 4-bit, group size 64, is the default; 32 tensors keep more bits, all with group size 64: embed_tokens at 8-bit, eight v_proj and one down_proj at 6-bit, and 22 attention and down_proj tensors at 5-bit. lm_head stays at 4-bit.
Tensor payload	19,457,774,524 bytes / 18.12 GiB, about 4.70 bits per weight with scales
Safetensors shards	4
Calibration	oqe_code_multilingual, 128 samples × 512 tokens (importance matrix)
Context in the config	524,288 tokens
Intended runtime	oMLX on Apple Silicon/macOS

Choose a variant

Variant	Nominal tier	Tensor payload	Bits per weight	Shards
oQ4e (this repo)	4-bit	19,457,774,524 bytes / 18.12 GiB	4.70	4
oQ6e	6-bit	27,732,284,618 bytes / 25.83 GiB	6.70	6
oQ8e	8-bit	35,189,757,270 bytes / 32.77 GiB	8.50	7

No quality, memory, time-to-first-token or throughput comparison on one Mac is published here; the table compares sizes only.

Memory for long contexts

Besides the weights, the KV cache needs memory, and it grows with the context. At fp16 it takes 208 KiB per token (52 layers × 8 KV heads × 128 dimensions × keys and values × 2 bytes) in all three variants:

Context	KV cache at fp16
32K	6.50 GiB
128K	26.00 GiB
512K (the config's maximum)	104.00 GiB

The numbers are estimates from the config for one sequence without KV-cache quantization. The runtime adds its own buffers.

Download

Install the Hugging Face CLI, then put the complete repository below oMLX's model directory:

mkdir -p "$HOME/.omlx/models/pyros-vault"
hf download pyros-vault/Nanbeige4.3-33B-oQ4e \
  --local-dir "$HOME/.omlx/models/pyros-vault/Nanbeige4.3-33B-oQ4e"

Serve with oMLX

Install the current oMLX runtime and start its OpenAI-compatible server:

brew tap jundot/omlx https://github.com/jundot/omlx
brew install jundot/omlx/omlx
omlx serve --model-dir "$HOME/.omlx/models"

Look up the model ID your oMLX version exposes:

curl http://127.0.0.1:8000/v1/models

Then use that ID with the OpenAI-compatible endpoint.

Chat template and sampling

The bundled chat template is the upstream one. The Nanbeige team describes its arguments:

• enable_thinking turns the reasoning on (the default); set it to false for answers without reasoning.
• preserve_thinking keeps the reasoning of earlier assistant turns. The team recommends false for chat and questions, and true for multi-turn tool use, office tasks and coding agents.
• With tools, the template switches to tool use; tool_call_format="xml" gives the best tool calls, json also works.

Whether a request can set these arguments depends on your oMLX version and client.

Upstream's sampling recommendations:

Scenario	Temperature	Top-p	Max new tokens
Agentic and tool-use tasks	1.0	0.95	65,536
Reasoning and chat	0.6	0.95	262,144

Quantization and verification

The bundled oq_imatrix_report.json records calibration with oqe_code_multilingual over 128 sequences of 512 tokens. It lists 364 importance entries applied to 364 linear modules, one missing name (lm_head), and no shape mismatches. All three variants use the same importance matrix. It was computed for this build and reused for oQ6e and oQ8e.

This card rests on the repository's structure, configs, shard count, payload size and quantization metadata. It makes no claim about speed, memory use, long-context behavior or output quality on a Mac.

Limitations

• MLX/oMLX targets Apple Silicon and macOS; this repository does not run through CUDA.
• Hugging Face hosted inference does not serve this oMLX layout.
• The 524,288-token context is the config's value, not a claim that this context was tested or fits your machine (see the KV-cache table above).
• Reasoning, tool calls, memory use and throughput depend on the oMLX version, the client, the prompt and the Apple hardware.
• Quantization can change output quality. Evaluate this variant on your own workload.

License and credits

The Nanbeige team released Nanbeige4.3-33B under Apache-2.0; this quantization keeps that license. For questions about the model itself, see the source model card, and cite it as the Nanbeige team asks there.

Quantized and packaged by pyros-vault with oMLX/oQ.