It's supported now 👍🏻

aigirlfriend555 That's good i'll try it out. Thanks for valuable info

aigirlfriend555 弱弱的问一句 我30708g+16g共享，能用么

You have to launch ComfyUI without the "--use-sage-attention" flag in the .bat file.

and without the "--fast" flag, if it is not yet fixed.

cluster1500 thx dude

I believe it would be Q6 for speed, FP8 for quality, and Q8 for something in between.

For a 4090, fp8 will always be better than gguf. The gguf models are for people who have low VRAM since the model is loaded on the cpu and not the gpu. But that causes the generation of the image to be much slower since the CPU is not as good for that type of calculations. On the other hand, the .safetensors models (fp8, fp16) are loaded on the GPU.

@creatumundo399 don't tell what you don't know. All gguf can be loaded in gpu too

@zids The gguf model was created to be able to run models on PCs with less VRAM. Yes, okay, you can load it onto the GPU. But why would you download a gguf model when you have a 4090 when you can use fp8 or fp16 and even fp32, which work directly on the graphics card, faster and more efficiently? I may not know everything, but I see that I know better than you how to use the models.

fp8 will run blazing fast.

Not sure about Qwen, but in Flux Q8_0 model definitely better than fp8. But it slower than fp8

@ViktorIltimirov Yeah coz he is wrong on every metric, fp8's lose a LOT of quality in comparison with original/base model. Whereas a quant at the top of the shop @ Q8 , in the vast majority of cases are very close if not indistinguishable from OG base models. BUT yeah slower than FP8 of course.

Yeah, that's its huge con: the model seems to be trained for precision and it avoids creativity.
But since the model seems to be precise, in theory, the variety of keywords in the prompt compensate for the lack of variety from the seed change.
But then again, with some prompts it seems to lack training so it may perhaps be a very limited model.

I think the best way to get around this is to use some loras.

Wildcards.

I wonder if you're using bf16? I can only say that fp8 variant requires qwen_2.5_vl_7b_fp8_scaled With 16GB 4060 Ti it takes up to 7 minutes to generate an image. Your GPU should be about 15% faster.

@lyagushka420 5070 only have 12GB VRAM and Qwen-image fp8 need 19GB space (and around 2GB Clips). so it'll have a much worse experience compare to 4060ti with weaker GPU but much bigger VRAM.

I used GGUF-Q4KS and takes around 180 seconds to generate a picture with similar texture but much worse chinese text comprehension.

@lyagushka420 @uncinsane475 turned out to be user error. I was trying to use it as a Checkpoint but it should have gone in diffusion_model. I ended up going back to the OG fp8 version and I'm able to run 1328x1328 at 50 steps in about 5-7 minutes.

fp8 model can run with 12Gb VRAM. Image can generate less than one minute on 3080 RTX when it combined with lightning 8 step lora and cfg 1

"Old"