This was trained on Sabu's recent art. I tried to delineate between his monochromatic, his greyscale, and his more fully colored art, which is why I put three trigger phrases.
Training params for version 3: { "lr": 0.00015, "engine": "ai-toolkit", "epochs": 30, "ecosystem": "zimageturbo", "keepTokens": 0, "networkDim": 32, "numRepeats": 8, "resolution": 1024, "lrScheduler": "constant", "minSnrGamma": null, "noiseOffset": null, "networkAlpha": 32, "optimizerType": "adamw8bit", "shuffleTokens": false, "textEncoderLr": null, "flipAugmentation": false, "trainTextEncoder": false }
I had real trouble getting a lora of the same quality as the version 2. I'm trying to understand what makes a good style lora, and the default settings really made terrible loras. So I kept fiddling with the learning rate, the number of repeats, but I still feel like I'm shooting in the dark :/
Description
I'm not sure about my prompting for the images. I edited the FLUX dataset's prompts, but I'm not sure it was really done well. We'll see 😬
FAQ
Comments (7)
Love the way it does tits, but it's awful at genitals and hit or miss with faces. I assume this is due to limitations in training data.
I'm wondering if it's my prompts, they're just a little edited from the autogen, and reused from the FLUX lora that itself wasn't very good. If I do it again, I'll have to spend more time on that
@Gwenwithgreeneyes Might be an issue with my prompting now that I think about it. I'll test a caption style prompt instead.
@Gwenwithgreeneyes Ok, more thorough testing shows that the genital issue is the base model, I hadn't done Zturbo before, but it clearly doesn't have much NSFW training data. Caption style prompts help with the face, but not entirely, I'd think that more training images would help with that, but hard to say. Last thing I'd suggest is cropping all the watermarks off your training images, I hadn't noticed how common they were on my first round, but the only real way to avoid those, is to not include them in the dataset.
@Dorian_White I had wondered if it was ZTurbo censoring, because it's very specific, but I hope I can accomplish more with a larger dataset. Frankly, I've been trying to avoid going through all these images and putting in a prose prompt myself, but I think at this point I've put more effort trying to get around that than I would have just doing it myself :/
The next version will use the images I cropped for my Illustrious model, so hopefully the signatures will be less of a problem. But that might be a few days as I work through the prompts. I know that for FLUX a lot of people would just use tags anyway, but I don't know if that's best.
@Gwenwithgreeneyes FLUX can do tags, but the quality goes down drastically on the results, so I probably wouldn't do it that way. If you crop out the watermarks they should basically completely disappear, so I wouldn't worry too much about that if you've already got a cropped set. I'm impressed with the dedication to actually write captions for your training data, half the reason I stick with Pony based stuff is so I can just do a few primary tags and let the autotagger do the rest. Captions are a pain.
To my understanding (which isn't extensive), ZIT isn't actively censored or censoring like FLUX did, but simply doesn't have much data for nudity (maybe we'll get checkpoints when the full model is released?).
As for signatures, I found with my Christopher Rush style model that simply tagging "signature" where applicable (which was most images) was enough to make them not show up, and that was with varied styles and locations of signature (some were "Rush" with a year, some were "CR", others were a stylized "CR" logo). Don't know if it would work for ZIT though.