SMD
Collection
Simplifying and Improving On-Policy Distillation for Few-Step Diffusion • 19 items • Updated • 5
import torch
from diffusers import DiffusionPipeline
from diffusers.utils import export_to_video
# switch to "mps" for apple devices
pipe = DiffusionPipeline.from_pretrained("MiniMaxAI/MiniMax-H3", dtype=torch.bfloat16, device_map="cuda")
pipe.load_lora_weights("zjwfufu/SMD-MiniMax-H3-T2VA-8step")
prompt = "A man with short gray hair plays a red electric guitar."
output = pipe(prompt=prompt).frames[0]
export_to_video(output, "output.mp4")SMD-MiniMax-H3-T2VA-8step
8-step SMD for video and audio generation. The student adapters, standalone inference code and reproducible examples are included. Upstream base components are downloaded separately.
Training code · SMD collection
Example
A rider and copper dragon climb out of a canyon toward SMD-shaped clouds.
Seed: 42. Size: 1344 × 768. Frames: 124 at 24 fps. Steps: 8.
Full prompt and case settings · Generation receipt
Models
| Variant | Weights | Steps | Size |
|---|---|---|---|
| default | LoRA | 8 | 1.38 GB |
Run
For a native ComfyUI workflow on a single GPU, use the corresponding SMD-MiniMax-H3-T2VA-8step-ComfyUI. It includes a Compact profile and a BF16 transformer profile. The Python runner below uses two GPUs.
Base: MiniMaxAI/MiniMax-H3.
Use Linux, Python 3.12, Git and two NVIDIA GPUs with CUDA support. The requirements install Diffusers from a pinned Git revision. The included inference code does not import the SMD training repository.
python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install huggingface-hub
# Sign in to Hugging Face.
hf auth login
hf download zjwfufu/SMD-MiniMax-H3-T2VA-8step --local-dir SMD-MiniMax-H3-T2VA-8step --exclude "lora/adapter_model.safetensors"
cd SMD-MiniMax-H3-T2VA-8step
python -m pip install -r requirements.txt
python infer.py --output outputs
The runner requires two CUDA GPUs and launches two processes with FSDP2. It was tested on 2 × A100 80 GB, with 433 GiB of host RAM. It downloads only the selected transformer plus shared encoders, decoders and processor files to the Hugging Face cache. The selected base components occupy approximately 144 GB (134 GiB) per workflow; keeping both workflows reuses the shared components and requires approximately 210 GB (196 GiB) in total. Loading and fusing these weights can take longer than the sampling steps. --base selects a complete local upstream snapshot, and --case selects another case JSON.
The runner prefers smd_lora.safetensors at the repository root and falls back to the retained lora/adapter_model.safetensors path. Both files have identical bytes. The download command excludes the legacy duplicate; adapter fusion and sampling settings are unchanged.
Sampling
The released sampler uses raw final-segment scale 0.5. The raw grid is built before the model-specific time shift. The included runner follows sampling.json; changing only an upstream pipeline's step count does not reproduce the student.
Video shift is 12; audio shift is 3. LoRA scale is 1.0. The default case uses 124 frames at 24 fps and joint video/audio generation.
The supplied loader validates and applies all LoRA matrices and saved replacement tensors, with the recorded merge precision. Omitting replacement tensors or using a different base can change the result.
License and attribution
Base components retain the terms described in the MiniMaxAI/MiniMax-H3 model card. Original code notices and bundled licenses are retained. Adapter use remains subject to the applicable base-model terms.
- Downloads last month
- 210
Model tree for zjwfufu/SMD-MiniMax-H3-T2VA-8step
Base model
MiniMaxAI/MiniMax-H3