The checkpoint type is set to "Trained" because Version 3.0 has been finetuned. Version 2.0 and 1.0 were purely done by merging checkpoints though.
The baked in VAE is not in fp16 precision, so trying to load it with --vae-in-16fp will produce black images. If you want to use that feature, I would recommend the modified sdxl vae from https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/tree/main. The decoded images will be different, but the difference should only be minor.
Version 3.0
A finetune on top of a Merge of NoobAI_epsilonPred10 with Noobieaterv20. Softens out the harsh linework of the previous versions, while keeping the more saturated colors and higher amount of detail.
V3 was done in two stages. First, a similar approach as V2, where I merged NoobAI_v1.0 with Noobieater_v2.0 through the perpendicular component method, while keeping the NoobAI CLIP. I tried to enhance this merge by adding small amounts of other models, but found these to end up worse, so I stuck with the initial merge. That model was then finetuned for 3 epochs on a small dataset of curated artists and concepts (~1000 images). My goal wasn't to change a lot, but just to soften the Itercomp influence on the base model and I feel like it achieved that, and it is actually more aligned with BaseNoob in terms of composition and varience than the base Itercomp merge was. It's strange, but I'm happy it worked out that way.
Version 2.0
A merge based on NoobAI-XL (epsilon-pred 0.75-version). While losing some of the creativity in posing, it is still rather close and gains stronger colors as well as improved character coherency and even gets more consistent at generating hands and other details.
V2 was produced by trying several similar approaches as V1, such as adding the difference between Itercomp and SDXL to it, as well as the difference between V1 and NoobAIXL_Epsilon-pred 0.50-Version, and then merging the best versions with each other and the 0.75 version itself. I repeated this step a few times until I felt like any additional changes only worsened the outcome. I kept the 0.75 Text Encoder, with the goal of preserving the increase in characters it knows, and I feel like not only did it keep its knowledge, but stabilize it a bit as well.
The goal was again to retain the character knowledge, pose variety and general feeling of the base model, while enhancing it as much as possible, especially the colors. While I don't think I wholly succeeded in this, particularly in keeping the poses as varied as they were in 0.75, it's as close as I can get it while keeping the colors the way they are now.
Version 1.0
A stupid little merge based on NoobAI-XL (epsilon-pred 0.5-version). It is a bit worse with some artist styles, however I like the colors and feel like the coherency has been improved, so I am sharing it. I would classify it as a general sidegrade that's useful if you want more saturated colors.
To get here, I merged Noob with the perpendicular component of IterComp (https://huggingface.co/comin/IterComp), and then merged that with a blockwise merge of Artiwaifu Diffusion 2.0 and Illustrious. I kept the original NoobAI Clip and used a slightly altered VAE that I found on an anonymous mongolian basketweaving forum.
Thank you to reoe for the perpendicular component idea
Description
Merge of 1.0 and noobieaterv20, with an additional finetune on top
FAQ
Comments (17)
i'm mirin that 3.0 showcase
think you could post the recipe you used for the base before training?
Thanks for reminding me to check, cause I actually wasn't sure anymore which one was which. Turns out, I used the perpendicular component merge(the method way at the bottom) of Noob1.0 and nobieater2.0 as the base. I did try a lot of other stuff as well and on top of it, but I feel like Noob1.0 is just too far along in baking already, and most other models will just end up destabilizing it again.
Updated that in the description as well.
Also, I still gotta get around to actually testing your mix as well, seemed pretty promising from my quick comparisons though
@phoenician In case you're curious about the difference the finetune made, here's a comparison between the merge(alpha) and full model https://files.catbox.moe/xagj44.png.
And some more, with different names (alpha is 30 here and finetune is 31):
https://files.catbox.moe/4zcrtj.png
@jyrrata clock work me bredda
I'm getting pure black gens when using version 3.0, even though I can see how the actual image is generating (I can see previews of the gen when generating and it's not pure black, it's actually the right image, but the moment when the gen finally finishes generating, it's all pure black), I don't know what's the Issue. I swapped to base noob or even other SDXL models and it works fine. Using ReForge.
That's very strange. I just tried it on a fresh install of reforge and with the 3.0 version, and it produced a normal image. If it's after the generation, maybe it's a vae issue? Do you have it set to automatic? If so, I'd try reinstalling the model, in case it was corrupted. If that doesn't help, I'm gonna assume it's some other setting. In that case I'm not really gonna be any help cause I don't usually work with forge/reforge, sorry, maybe you could try asking on the NoobAI Discord server, or even on the reforge github page?
I did find one report of something similar, however it's over a year old and I don't know if it's applicable to you since it's auto and not reforge https://www.reddit.com/r/StableDiffusion/comments/16qazyg/black_images_are_back/
@jyrrata Found the issue. I forgot I was using --vae-in-fp16 on init args, this model outputs black images when I use it. When I turned it off, the image came perfectly fine. Sad, because --vae-in-fp16 makes my gens to be around ~15 seconds faster compared to when I gen without this arg.
@jyrrata I checked the issue report and it's giving me this when I gen with -vae-in-fp16: RuntimeWarning: invalid value encountered in cast
x_sample = x_sample.astype(np.uint8)
Maybe diffusers related?
@DefaultBar I'm glad you found the issue, but that's obviously not ideal if it's taking that much longer. It could be diffusers related. I guess maybe some weird conversion of the vae happened during the finetune that changed the vae's precision. You could try using the base sdxl vae from here for example https://huggingface.co/madebyollin/sdxl-vae-fp16-fix. While the baked in vae is not the base sdxl one, the difference is very minor. Or you could extract the vae from noob1.0/2.0 and use that one.
@jyrrata Yep, it was the vae thingy. Fully working now, using the fp16 fix. It looks different but it's not a big difference. Thank you.
@DefaultBar I just realized I also misunderstood what you meant when you said you tested on base noob. Thought you were saying on one of the other noobieater versions, and got confused when I just tested the vaes and they were exactly the same and had issues. Now I get you mean just noobai, and now everything makes a lot more sense.
I looked again at where I got the VAE, and the creator even said that it's not fp16 (Someone even had the same issue as you, I must have missed that when I found it). Will do a disclaimer in the description, but yeah it seems the vae will just not work on fp16 for any noobieater version
v3.0 is awesome! I used it in a merge, if you don't mind. The contrast is improved.
@reoe I don't mind at all. Especially since I got the itercomp idea from you in the first place. Happy to hear it has a positive effect on V2
Hi :3
in the prompts, you mention several names of artists, improving the quality of the image, if you don't mind, where can I see those artists, I'm new at this xD? Is there a hidden list somewhere?
Hey there.
This model uses the same artists as the base NoobAIXL model, which are danbooru tags, so you can find artists there.
If you mean a comprehensive overview of all possible artists, there are a few, but Panchovix has spent some time comparing 35k artists, sorted by image count on danbooru. https://huggingface.co/datasets/Panchovix/noob_1.0_top35kartists/tree/main . The images are kind of big, so you might also be interested in something like this https://huggingface.co/datasets/Takenoko3333/danbooru-artist/blob/main/wildcard/artist/artist-100up.txt. That one only has text, so it's easier to load, but you'd need to figure out if the style of the artists is what you want on your own.
@jyrrata thanks :3/ have nice day :)
Details
Files
Available On (1 platform)
Same model published on other platforms. May have additional downloads or version variants.