Having FP8 would be nice.

But GGUF Q8_0, would be better.

It's much closer in quality to the full size model, and about the same size as FP8. 

https://huggingface.co/city96/FLUX.1-dev-gguf/tree/main

@jr81 I know, but fp8 can do much faster generations wirh 4000 series cards (fast flux optimisations)

It's already there ! It's the 22 GB one.

What we're missing here is a GGUF Q8_0. The one we have is Q4_0, I think. 

Hey Q4 is incredibly useful too, so best to have it too

Yeah, Q4_0 is good quality for its size.

I also use it, along with Q8_0 :)

Without B it would be Pineapple.

Agree. Corporations get away with far too much.. They always steal from everyone, customers, people working for them. It's a big systemic scam, and everyone below the top is gettin played.

All AI images are stolen from Artists. Fuck AI

What evidence do you have??? You realize right all AI images are stolen from real Artists

@KingArthur88 Technically not true. I mean, in terms of art content, yea. But there's also a lot of RL images. I get where you're coming from, cuz I had a bad attitude towards this stuff before I got into it. And I agree that it's had a bad effect on art overall. It's gonna destroy artistic ambitions in the future. Hell, I was just getting halfway decent myself on my graphic tablet, but there's not much sense in putting in all that effort when it can be done with a generator.

I suppose if anything, real art will eventually become, real artists training their own models/loras and selling those with special license to reproduce their styles.

Well, if you really want to be mad at anyone, get mad at those responsible for AI being used to replace artists and musicians instead of CEO's and greedy/corrupt politicians.

@KingArthur88 well try rendering the "Lays Chips" . You can see that the results are almost 99.99% identical from the images rendered with "Lays Chips SDXL Lora" using SDXL. They have obviously stolen the data from this and merged it into their own FluxDEV model. How is it legal for them to steal this LORA Data and then make a "You cant use it hehe"-license on it?

@cathyleverman One word, Lobbying. The wealthy make the rules. It's like letting a five year old make the rules when playing monopoly. No one is gonna win/have fun but the kid. Same concept for the wealthy elite under capitalism(AKA; socialism for the wealthy). Have you actually read legal paperwork? It's practically in Chinese. Cuz if any normal person could understand it, a lot of people would stand up and go 'wtf is this shit..?' The legal system is built in such a way to not be understood by the common man.

@Lazman its quantum law. You are both allowed and not allowed to do anything, depending on what whims the judge will sentence you on :D

@cathyleverman Ain't that schrödinger;s cat?

@Lazman it would be fun to find what other LORAS they have stolen and then rebranded in a restrictive license that benefits only them.

This type of license fraud and license terror seems to be quite a common new behavior in the Generative AI world.

@cathyleverman Corporations and the wealthy stealing the work of those with no money and/or those that work for them and branding it as their own, is nothing new. Steve jobs didn't invent the Iphone, he had no damn part in it, and when the idea was first brought to him, he rejected it. But the prick acted like he made it himself. It wasn't even Apple that made the thing, not entirely. It was a company that they bought up, which probably only went under in the first place cuz of corporations like Apple controlling the market and pricing them outta business. Jobs didn't even invent the Macintosh, that was Wozniak, jobs just took all the credit, that guy was just a sleazeball in a turtleneck.

@Lazman pretty much

@cathyleverman That isn't to say we should give up on raising awareness of such practices though. People have grown unbelievably numb to corporate control. In 1984, people in America literally rioted in the streets, I mean like, filling the streets with people, over their distaste for the flavor of 'new coke'. These days, the richies double our rents in just a few years, and the most I've even seen is a parade of like, 30 people bashing Trudoh.

Corps and the gov have learned to master a mix of psychology and diplomacy, and it's almost impossible to fight against the passive aggressive nature of it all..

The 22 GB mode is UNET, ( in FP16 ).

Sorry, should of mentioned FP8 would be good. It means your model will work well in 16gb, possibly 12gb of vram at a decent quality and speed. Gguf is ok , but not as fast as FP8 and possibly lower quality. Your V1 unet only model is 11Gb and works really well.

I've just managed to convert your FP16 V2 to an FP8 version using a python script ..so far it seems to work well.

@jr81 weird because forge asks for T5/clip etc when i use the 22gb one

@Nmdrcn The UNET model requires T5/clip and vae.

A "checkpoint" has all included ( at least that's the term used in ComfyUI ).

But I think there are only Flux checkpoints in FP8 ( about 17GB )

@jr81 forge is different. only |unet only| models don't seem to ask for t5/clip and vae. This 22gb model asks for them regardless

same happens to me

+1

It worked in ForgeUI for me, I use a 12GB GPU (RTX 4070 Ti)

Take your GPU RAM and subtract 4096 of its RAM on the GPU Weights. In my casa, I took 4096 out of 12288 on the GPU Weights. Then set Swap Method to queue and Swap Location to CPU.

My computer has 32GB of RAM as well and I added more 32GB of RAM as virtual memory. Sometimes it crashes, but it works most of time.

I would also like to know the official settings for this model.

But also, what custom settings do the users use.

What quantization is the current gguf? I usually use a 5K_S or 5K_M

This is probably Q4

You can tell by the file size.

Here are all Flux dev quants, by the ComfyUI-GGUF extension developer ( take a look at the file sizes ):

https://huggingface.co/city96/FLUX.1-dev-gguf/tree/main

@jr81 Thank you!

@Sa_May Please publish it :-) :-(

@Sa_May ...oh! yes please, publish it. v1 and v2 are showing a great trend :-)

Is it possible to quantize models like this one using llama.cpp? It has a mini program that quantizes models to q_whatever, but I've only tested it with LLM so I'm not sure if there's a different process for diffusion models.

@jtabox From what I've read it's similar or the same to quantizing llms, which doesn't take long. I can definitely do it its just that the base models are so big it's annoying for me to download, but overall yeah I'm just lazy. (but for the average civitai user they for sure won't do it themselves lol). This site will soon have its own version of "TheBloke" or "bartowski" soon I bet.

I'm a noob when it comes to stuff like this, and I managed to do it for another checkpoint using this guide.

depending on your level of expertise. Note you can import directly from Civita using the d/l URL too.

you can quantize this or any other model yourself using the optimum.quanto library or bitsandbytes. Here's an example:

from diffusers import FluxTransformer2DModel, SD3Transformer2DModel

from diffusers import UNet2DConditionModel

from optimum.quanto import freeze, qint8, quantize, quantization_map

from pathlib import Path

base_model = 'C:/Users/xxxxx/.cache/huggingface/hub/Civitai Models/fluxRealistic_fluxRealisticSamayV2.safetensors'

dtype = torch.bfloat16

transformer = FluxTransformer2DModel.from_single_file(base_model, subfolder="transformer", torch_dtype=dtype)

quantize(transformer, weights=qint8)

freeze(transformer)

save_directory = "./flux-dev/fluxRealistic/fluxtransformer2dmodel_qint8"

transformer.save_pretrained(save_directory)

qmap_name = Path(save_directory, "quanto_qmap.json")

qmap = quantization_map(transformer)

with open (qmap_name, "w" , encoding= "utf8" ) as f:

json.dump(qmap, f, indent= 4 )

print('Transformer done')

Okay, what difference will Q8_0 make? I checked the Flux GGUF models. Q8_0 still is big and needs higher amount of VRAM. I don't understand how you can be in the middle. I have a 12 GB card and the FLUX GGUF Q8 slows down my PC like hell. I have to just shut it down. From 3 days I am not having any luck of generating anyting good or sexy.

@NOOBDA The next logical step is to scale down to like nf4. (gguf or bitsandbytes or quanto) to reduce. your choice.

bro you need downloads flux's vae and comfyui gguf node

If you get the SafeTensor run it by itself since it's baked in. If you get the GGUF then make sure to have your vae and clip models (T5 & ClipL) loaded in the text encoder box

***Edit: if you are running Forge***

bro you need flux‘s vae and clip

@a114514lh929 "This is Checkpoint. CLIP and VAE is already on" so we still need one? im using the 10 g version

you should change the sampler to euler + simple

@redforce770387 "AssertionError: You do not have CLIP state dict!" XwX

@Sa_May is it applicable for the nf4 model also?