NVFP4?
#2
by
ktsaou
- opened
@Firworks it would be amazing if your could convert this model to NVFP4 !
btw, I run minimax-m2 on 2x nvidia rtx 6000 pro blackwell, and it is the extremely reliable and performant. Minimax-M2 is gold for rtx blackwell. I hope this one will be too.
I've spent some time today attempting it but I think I'll have to do a few monkey patches to llm-compressor to get it to run to completion. Hopefully I can get it run. I spent a while trying to get the original M2 run as well but stopped when someone else successfully published an NVFP4 quant of it. The M2 NVFP4 quant was done with ModelOpt so using a different process than I normally run.