bdbj
/

Llama-3.1-8b-qpal-df-msq-fig1d

Model card Files Files and versions

Model Card

Base model: meta-llama/Llama-3.1-8B
Quantization method: Latency constrained fusion-aware MSQ with Q-Palette
Backend kernel: Q-Palette kernel
Calibration data: N/A (data-free)
See Figure 1 (d) of our paper.

How to run

Follow the instruction in https://github.com/snu-mllab/Q-Palette.

References

Model Paper

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for bdbj/Llama-3.1-8b-qpal-df-msq-fig1d

Base model

meta-llama/Llama-3.1-8B

Quantized

(297)

this model

Collection including bdbj/Llama-3.1-8b-qpal-df-msq-fig1d

Data-free quantization w/ Q-Palette

30 items • Updated 26 days ago