Aozora-XL: A V-Prediction SDXL Model

Thanks. While it's certainly not as powerful as a full fine-tune, training on datasets of thousands often doesn't require retraining everything – most layers already possess significant knowledge. I see it as a middle ground between Dreambooth and a full fine-tune. Plus, with good hardware, you can use the same script for faster training by increasing the batch size.

@candy_69 

With 8GB of VRAM, you can remove the feed-forward layer (the "ff.net") to save some VRAM it takes up about half of the parameters that are trained. This might let you fit the model into 8GB, though it would be pushing the limits.

Be aware: disabling ff.net will reduce the model's ability to learn new concepts effectively. However, this approach should still yield better results than training a LoRA and merging it later.

If you're not comfortable coding, try pasting the script into an AI assistant and ask it how to disable specific modules.

but it think 12gb of vram is the absolute limit but the good news is a 3060 with 12gb used is around $150-200 so you could easily build a machine to train for cheaper than renting for a few hundred hours

@candy_69 I turned my script into a gui based script with auto installation, so this should allow you to just run it without needing to know anything code related, you can check it out on my github

How are you prompting for them? the model isnt finished with training so it will get better but its also trained from danbooru so it would be prompted like "1girl, d, bandeau, black jacket, burnice white". i can make continuation training for a more standard format if you provide it

For the next training run i will use non cleaned danbooru tags as it seems to be a issue, look out for my next release and character recognition for zzz should be alot better, it should be done training in a few days

I'm intentionally avoiding stylistic overfitting with this model. By selectively training only key UNET layers and minimizing changes to the base, I can mostly prevent catastrophic forgetting or overfitting. I'm still experimenting to find the optimal balance, which is a slow process.

The result is a model with high stylistic neutrality and flexibility. This makes it an ideal, non-opinionated base that should be highly receptive to LoRAs, unlike models that have been heavily fine-tuned for a specific look.

Alot of people like models with a baked in style but i find that i would much rather have a model i can slap a lora onto when i want it

@NunFish I’m currently improving my dataset to fix the model’s color issues. NoobAI has been flooded with so much Danbooru content that it’s hard to find images it hasn’t seen from the easiest place to get data. Many other fine-tunes overuse Danbooru data, reshowing images and causing overfitting. Colors should be better in v0.2, and afterward for 0.3 I’ll work on fixing the round faces that im sure you may notice.

it was trained at a max of 1152, but images weren't stretched or warped due to my bucketing system, most of them where weird resolutions like 500x1152, ect. so its could have reduced the guessing and warping you see from models at higher resolutions. but in no way was it trained to be specifically for 1536. but most images where on the larger size

but if you do some generations you will sometimes be mistaken and second guess if its a vpred, to be honest 100% i think training specific layers on the vpred foundation is the cause, for some odd reason its stabilized itself and im not 100% sure. it performs more like a eps than a vpred as its less muddy. you might even call it broken but in a good way. tho its not perfect but compared to other vpred models i use, i would say this model produces better hands and features 60-70% of the time compared to the 40-50% of other models