Can we perform 4-bit quantization for the awq of the Step-3.5-Flash model? The VLLM can run it.

by lsm03624 - opened 29 days ago

As the title says

cyankiwi org 28 days ago

Thank you for your interest. Step-3.5-Flash is in calibration, and its AWQ version should be available soon!

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment