SmolLM3-3B-q4f16_1-MLC

This is Hugging Face's SmolLM3 3B model in MLC format q4f16_1. It supports text generation in WebLLM.

The package context window is 65,536 tokens, with a prefill chunk size of 1,024. The proposed WebLLM record defaults to 4,096 tokens to limit memory use. The q4f16_1 variant requires WebGPU's shader-f16 feature. Matching WebGPU candidates use v0_2_87 and web-runtime 0.28.0-dev2; see library provenance.

Reasoning is on (the HF default). Turning it off needs a package generated with --conv-template smollm3_nothink; enable_thinking does not apply. The /think, /no_think and /system_override system-message switches of the HF template have no effect. The HF template writes the current date into the system prompt. This package's template omits that line. Sampling defaults come from the upstream generation_config.json: temperature 0.6, top_p 0.95.

Conversion and WebGPU sources

  • Model: a07cc9a04f16.
  • TVM release base: be03dc94b001; used for weight conversion; the local Metal timer patch is recorded in the build information.
  • Weight-conversion MLC-LLM: 5e824377af7f.

Weights were converted with the sources above; WebGPU libraries were compiled separately with the source revisions in the linked library provenance.

Reproduce the conversion

With the conversion toolchain above installed and the pinned HF revision downloaded:

mlc_llm convert_weight hf/SmolLM3-3B --quantization q4f16_1 --device cpu -o dist/SmolLM3-3B-q4f16_1-MLC
mlc_llm gen_config hf/SmolLM3-3B --quantization q4f16_1 --conv-template smollm3 --context-window-size 65536 --prefill-chunk-size 1024 -o dist/SmolLM3-3B-q4f16_1-MLC

These are equivalent regeneration commands. The shipped configuration was retained from the original generation and adjusted to the maximum context and 1,024-token chunk; its SHA256 is recorded below in the build information.

mlc-build-info.json records the full revisions and source weight checksums. See the upstream model card for model capabilities, evaluations, and limitations, and the WebLLM documentation for runtime usage.

Downloads last month
50
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for mlc-ai/SmolLM3-3B-q4f16_1-MLC

Quantized
(139)
this model