How to use from
SGLang
Install from pip and serve model
# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Vortex5/Stellar-Seraph-12B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Vortex5/Stellar-Seraph-12B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'
Use Docker images
docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Vortex5/Stellar-Seraph-12B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Vortex5/Stellar-Seraph-12B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'
Quick Links

Stellar-Seraph-12B

Overview

Stellar-Seraph-12B was created by merging Wicked-Nebula-12B, Crimson-Constellation-12B, Celestial-Queen-12B, and Darklit-Maiden-12B, using a custom method.

Merge Configuration
models:
  - model: Vortex5/Wicked-Nebula-12B
  - model: Vortex5/Crimson-Constellation-12B
  - model: Vortex5/Celestial-Queen-12B
  - model: Vortex5/Darklit-Maiden-12B
merge_method: nexus
chat_template: auto
dtype: float32
out_dtype: bfloat16
tokenizer:
  source: Vortex5/Wicked-Nebula-12B

Intended Use

โœง
Roleplay Emotion-forward interaction
โœฆ
Storytelling Structured long-form narrative
โŸก
Creative Writing Atmospheric fiction
Downloads last month
10
Safetensors
Model size
12B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for Vortex5/Stellar-Seraph-12B