nvidia
/

GLM-5-NVFP4

Text Generation

Model Optimizer

8-bit precision

Model card Files Files and versions

zhiyucheng commited on Apr 6

Commit

3faeeee

·

verified ·

1 Parent(s): 8b94a19

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -106,7 +106,7 @@ This model was obtained by quantizing the weights and activations of GLM-5 to NV
 ## Usage
-To serve this checkpoint with [SGLang](https://github.com/sgl-project/sglang), you can start the docker `lmsysorg/sglang:nightly-dev-cu13-20260305-33c92732` and run the sample command below:
 ```sh
 python3 -m sglang.launch_server --model nvidia/GLM-5-NVFP4 --tensor-parallel-size 8 --quantization modelopt_fp4 --tool-call-parser glm47 --reasoning-parser glm45 --trust-remote-code --chunked-prefill-size 131072  --mem-fraction-static 0.80

 ## Usage
+To serve this checkpoint with [SGLang](https://github.com/sgl-project/sglang), you can start the docker `lmsysorg/sglang:nightly-dev-cu13-20260305-33c92732` and run the sample command below (when the nightly docker becomes unavailable, use `lmsysorg/sglang:latest`):
 ```sh
 python3 -m sglang.launch_server --model nvidia/GLM-5-NVFP4 --tensor-parallel-size 8 --quantization modelopt_fp4 --tool-call-parser glm47 --reasoning-parser glm45 --trust-remote-code --chunked-prefill-size 131072  --mem-fraction-static 0.80