Please re-download int8 version. I have uploaded much faster, proper version.
Versions
int8: recommended. Fast, accurate, compatible with almost any GPU.
mxfp8: added for comparison. In theory (and according to nVidia PR) should be more accurate than int8, but in practice I was not able to spot any definitive advantages. A bit slower than int8, but still faster than original bf16. Compatible only with RTX 50xx series (Blackwell).
Performance on my setup
original bf16 (baseline): 2.20 it/s +0%
int8: 3.23 it/s +46%
int8 + torch compile (comfy core): 3.59 it/s +63%
mxfp8: 2.58 it/s +17%
This is high quality int8 quantized version of WAI-ANIMA model. It retains ~90% of original model quality, but uses about 50% less VRAM and also runs faster on almost any nVidia GPU (AMD not tested). Nice trade-off, especially for low-end GPUs.
Can be used as a drop-in replacement for original model in latest ComfyUI, no custom nodes required. If you have troubles running the model make sure that you updated both ComfyUI itself and its dependencies (e.g.pip install -U -r requirements.txt on manual linux install).
Converted to int8 / mxfp8 using convert_to_quant script.
Description
int8, ConvRot group size 256, rowwise, learned rounding SVD
Proper, fast version.
FAQ
Comments (2)
thanks! it's much faster than the experimental version. however, for some reason, the compressed model is always 1% slower than the native bf/fp16 model quantized to int8 convrot on-the-fly...
somehow int8 run slower than mxfp8 (and original) on my end, up to +50% of time needed per image. On the other hand mxfp8 perform on par with original, slightly faster too.



