Instructions to use IlyasMoutawwakil/tiny-random-Muse-Spark-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use IlyasMoutawwakil/tiny-random-Muse-Spark-NVFP4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="IlyasMoutawwakil/tiny-random-Muse-Spark-NVFP4")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("IlyasMoutawwakil/tiny-random-Muse-Spark-NVFP4")
model = AutoModelForMultimodalLM.from_pretrained("IlyasMoutawwakil/tiny-random-Muse-Spark-NVFP4", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use IlyasMoutawwakil/tiny-random-Muse-Spark-NVFP4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "IlyasMoutawwakil/tiny-random-Muse-Spark-NVFP4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "IlyasMoutawwakil/tiny-random-Muse-Spark-NVFP4",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/IlyasMoutawwakil/tiny-random-Muse-Spark-NVFP4

SGLang

How to use IlyasMoutawwakil/tiny-random-Muse-Spark-NVFP4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "IlyasMoutawwakil/tiny-random-Muse-Spark-NVFP4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "IlyasMoutawwakil/tiny-random-Muse-Spark-NVFP4",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "IlyasMoutawwakil/tiny-random-Muse-Spark-NVFP4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "IlyasMoutawwakil/tiny-random-Muse-Spark-NVFP4",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use IlyasMoutawwakil/tiny-random-Muse-Spark-NVFP4 with Docker Model Runner:
```
docker model run hf.co/IlyasMoutawwakil/tiny-random-Muse-Spark-NVFP4
```

Tiny random Muse-Spark NVFP4

Random testing weights, initialized with seed 0. This model is intended for loading, inference, and serialization tests.

The tiny text model has two sparse decoder layers. Layer 0 stays in full precision; layer 1's routed experts use NVFP4 with group size 16. Attention and the language-model head stay in full precision. The producer metadata preserves the original model's exclusion policy, including entries for layers absent from this tiny model.

Quantized experts use the supported fused ModelOpt layout: packed gate_up_proj and down_proj weights, affine E4M3 *_weight_scale grids, FP32 *_weight_scale_2, and per-expert *_input_scale globals. No internal swizzled scale artifacts are stored.

The reference/ directory contains the matching unquantized BF16 model. This replacement fixture uses the current quantizers and supersedes an obsolete internal layout; it is not an evaluation of a pretrained Muse-Spark model.

Downloads last month: 86

Safetensors

Model size

653k params

Tensor type

F32

BF16

F8_E4M3