GGUF format?
#3
by
eqco
- opened
Hi,
I would love to test this model out but a quantized version would be the only way for me to run it on my setup. Any chance we could see one uploaded in the future?
Thanks!
Hi, yes, we plan to upload some quantized versions soon after we do some evals on the different quantizations. Any specific format you are looking into?
I think Q4_K_M would be great but anything around that could work too. Not sure how that will effect the model but just being able to test it would be worth it.