INT8 Mix Quantization of One Obsession v24 by maxfeifei8, I got really curious about how INT8 quantization would fare with Illustrious. Turns out pretty good.
70 layers in INT8 and rest in INT8 Convrot for the diffusion model.
v2.0
Text Encoders are now quantized with a mix of INT8 and INT8 Convrot, total checkpoint size is now down to 3.92 GB
A patch is required to run the model as ComfyUI does not support quantization methods for the SDXL Clip Loaders. YOU WILL GET GARBAGE OUTPUT WITHOUT IT.
Same prompt and seed from v1.0 so you can compare the images in terms of quality
v1.0
Text Encoders are untouched
Checkpoint runs natively on ComfyUI
Clone the below repo into your ComfyUI's custom_nodes folder. No need to add any nodes, it initializes and patches the CLIP Loader at runtime, please let me know if you encounter any issues.
https://github.com/bakapotatolord/ComfyUI-PotatoForge-Encoder-Quant
Sample images once again just the first stage (T2V), used prompts from the original model.
Tested on:
NVIDIA GTX 1660 Super 6 GB
32 GB System RAM
Test settings:
25 steps
CFG 4.5
Euler Ancestral / Normall
696 × 1048 resolution
Approximately 2.5 s/it on a GTX 1660 Super, about 45% faster than the usual model
Description
Quantized the text encoders (CLIP L and Clip G), bringing down the size to 3.92 GB
As ComfyUI does not support quantization methods for the SDXL Clip Loader, a patch is required, just clone the below link into custom_nodes/
https://github.com/bakapotatolord/ComfyUI-PotatoForge-Encoder-Quant
No node or anything required, patches during initialization