Instructions to use mlc-ai/SmolLM3-3B-q4f16_1-MLC with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLC-LLM
How to use mlc-ai/SmolLM3-3B-q4f16_1-MLC with MLC-LLM:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
SmolLM3-3B-q4f16_1-MLC
This is Hugging Face's SmolLM3 3B model in MLC format q4f16_1.
It supports text generation in WebLLM.
The package context window is 65,536 tokens, with a prefill chunk size of 1,024. The proposed WebLLM record defaults to 4,096 tokens to limit memory use.
The q4f16_1 variant requires WebGPU's shader-f16 feature.
Matching WebGPU candidates use v0_2_87 and web-runtime 0.28.0-dev2; see library provenance.
Reasoning is on (the HF default). Turning it off needs a package generated with --conv-template smollm3_nothink; enable_thinking does not apply. The /think, /no_think and /system_override system-message switches of the HF template have no effect.
The HF template writes the current date into the system prompt. This package's template omits that line.
Sampling defaults come from the upstream generation_config.json: temperature 0.6, top_p 0.95.
Conversion and WebGPU sources
- Model:
a07cc9a04f16. - TVM release base:
be03dc94b001; used for weight conversion; the local Metal timer patch is recorded in the build information. - Weight-conversion MLC-LLM:
5e824377af7f.
Weights were converted with the sources above; WebGPU libraries were compiled separately with the source revisions in the linked library provenance.
Reproduce the conversion
With the conversion toolchain above installed and the pinned HF revision downloaded:
mlc_llm convert_weight hf/SmolLM3-3B --quantization q4f16_1 --device cpu -o dist/SmolLM3-3B-q4f16_1-MLC
mlc_llm gen_config hf/SmolLM3-3B --quantization q4f16_1 --conv-template smollm3 --context-window-size 65536 --prefill-chunk-size 1024 -o dist/SmolLM3-3B-q4f16_1-MLC
These are equivalent regeneration commands. The shipped configuration was retained from the original generation and adjusted to the maximum context and 1,024-token chunk; its SHA256 is recorded below in the build information.
mlc-build-info.json records the full revisions and source weight checksums.
See the upstream model card for model capabilities, evaluations, and limitations, and the WebLLM documentation for runtime usage.
- Downloads last month
- 50
Model tree for mlc-ai/SmolLM3-3B-q4f16_1-MLC
Base model
HuggingFaceTB/SmolLM3-3B-Base