a distillation lora for anima, trained on ~100 images at 1024x generated by the model itself using 40 steps cfg = 4 er_sde from diverse captions
baked-in negative prompt: worst quality, low quality, score_1, score_2, score_3, blurry, jpeg artifacts, sepia
may or may not be underbaked/overbaked
works best with cfg = 1 and 8 steps for heun/2s samplers, 16 steps for 1s or multistep samplers, but you can use however many steps you'd like.
the training code is a modification of diffusion-pipe for RL/distillation + custom comfyui nodes for data generation
TODO: get a larger more diverse RL dataset and filter it for artifacts
TODO: do larger ranks and 1536x training, as well as auxillary loss(es)
Update: I may have been doing dataset generation wrong the entire fucking time... ComfyUI be damned. Possible new version soon(?)
Description
epoch 50 on 100 images, continued from epoch 15 but with grad accum 2
FAQ
Comments (5)
really amazing work. so far er_sde and simple/normal have worked best with 16 steps and 1 CFG.
it also works with different styles (as in it doesn't have a style lock) but it is slightly more prone to deformations (not hating, just in case you are working on a V2)
uploaded v2, this time i filtered the training set for deformations so it should work better
doing the god's work. its very much a improvement over the v1.
its also a very good improvement on character identities. v1 mixed them up a lot.
amazing work. one thing i found that it sometimes does "x-ray" shots without prompting for it. (which isn't particularly a bad thing, some might call it a improvement, and its like 1 in 4 shots so not that annoying either)
as for the anatomy, its much better than v1, but still struggles when there are 2 or more people, altough for single person images, anatomy issues are pretty much gone in v2. (again, not hating, just "constructive criticism")
@Hycros ep100 does sometimes give me overfit vibes so i may upload ep80 of v2. As for v3 I'll expand the dataset maybe around 2x (if i don't go insane while filtering images) and rank up to 64.
Additionaly I would like to implement ArcFlow training and sampling in comfyui but it's pretty difficult 🫠
@levzzz another thing i would like to point out is that in most images, details feels washed (slightly). kinda like someone adding negative weights on a detailer lora.
i have been doing CFG 1.5 and 8-10 steps with res_multistep+simple. this has mostly fixed everything and now i get almost same quality (80% of the gens) as the base model in 1/3rd of the gen time.
i just wanna say thnak you for this amazing work. may both side of your pillow be forever cold😜. looking forward to what you do with it in future.