vae: mixed fp16 decoder + fp32 encoder/scales, single shard
#1
by OzzyGT HF Staff - opened
Replaces the 3-shard fp32 video VAE with a single mixed-precision checkpoint: decoder weights in fp16, encoder + quant convs + the per-block scale1/scale2 residual anchors kept in fp32.
Loads correctly with current diffusers (the class's _keep_in_fp32_modules upcasts the decoder, so behaviour is unchanged). Narrowing that list to ['encoder', 'quant_conv', 'post_quant_conv', 'scale1', 'scale2'] gives 5.19 GiB of weights and 6.06 GiB of decode peak instead of 9.70 / 14.63, at 77.2 dB vs 77.3 dB against a true-fp32 decode (81.1 dB against the previous checkpoint, max deviation 0.78/255).
Download shrinks from 9.8 GB to 5.2 GB. See the README.
OzzyGT changed pull request status to merged