Universal CLIP
You can use any universal CLIP, the "base" refers to the preferred base model
PONY_Base When Used in:
Pony - Full PONY character retention & SDXL name recognition (See Seed to Seed)
illustrious - Do not use PONY_Base-CLIP-L in illustrious
SDXL - Working Likely improves NSFW on models trained with that goal.
FLUX - Likely improves NSFW with minor loss on TEXT recognition
SD 1.5 - Not Working
SD 3 and SD 3.5 - untested
illustrious base:
May only work with the CLIP-G and in certain models.
illustrious - Paired with CLIP-G characters should prompt.
Pony - Working on Most PONY Characters but not FUTA
SDXL - Working Likely improves NSFW on models trained with that goal.
FLUX - Likely improves NSFW with minor loss on TEXT recognition
SD 1.5 - Not Working
SD 3 and SD 3.5 - untested
Single CLIP Loader
SD 1.5
Dual CLIP Loader
For FLUX use any T5 and either Universal CLIP-L (Pony_base has been tested more)
For SDXL you can use either universal base CLIP-L model with the Pony Universal CLIP-G
For PONY use Pony Base Universal CLIP-L and Pony Base G
For illustrious use Illustrious Universal and and Pony Base CLIP-G (Universal illustrious CLIP-G is still training)
Triple CLIP Loader
SD 3 & 3.5
Description
FAQ
Comments (15)
for example in illustrious merges should i use this or joy?
if your attempting to merge illustrious as the primary then the clip here illustrious base, but the character embedding are almost non existent in the universal base allowing it to merge - in the joy illustrious those clips still have them intact
One question. I usually use Noobai XL V-Pred 1.0 version. Will it improve my accuracy if I use it?
Not sure on that
YOOOO THANKS FOR THE ILLUSTRIOUS ONES.
VERY USEFUL.
For the IL CLIP, it has a great positive prompt adherence but tends to ignore negative prompts
Interesting amitedlty with flux I got used to positive only prompting and did not test negative much
It's a general problem with IL (sdxl) though. I've swapped completely over to NegPip. I keep negative prompt with defaults like: lowres, worst quality etc. - but then for things I specifically do not want I just do (unwanted-thing:-1.0) in positive prompt. or higher/lower, depends how stubborn the model is. available for all the important web UIs.
I have a question about clip. When comfyui wants to operate on clip, should it discard the original clip of the model directly or use the new clip L+G to merge with the original model (addition, proportional fusion)?
So comfy UI uses the vocab and config it has stored to load the weights from the files. For dual clip it would over rife the merged clip fully when connected to positive negative or a Lora + conditioning
First of all, thank you for your work on the CLIPs. 👍
After reading all your CLIP descriptions, I still can't decide which one is better at a specific stuff than the others. Could you add a very clear explanation of each CLIP's strengths compared to the others?
For example, if one is more suitable for realistic content, please specify. If one is more suitable for anime content, please specify. If one is more suitable for SFW content, please specify. If one is more suitable for NSFW content, please specify.
More complete descriptions of each CLIP's strengths would be a significant improvement.
They are not focused in that way as the dataset was diverse, while it might have more NSFW images then the LAION set that clip was trained on the primary focus is accuracy. An example of a fraction of the data used is the north american birds by name dataset I have at huggingface