Check out my Gumroad if you have any interest in buying specialty loras! https://satoulofthouse.gumroad.com/l/kvsoy
This is my first lora! I trained with anylora on colab.
I'll update this to improve it later, as this was just a test to see if it was possible with 28 pictures of her from episode 1. There's only portrait pictures, but as you can see, all you need to do is tell it you want a full body shot and put portrait in negatives and then just inpaint her face and hair on full strength and it will fix it.
Her hair pin will not get much better from inpainting at this time. It's a really simple fix that even a non artist can fix if you wanna just draw it in. In the anime, the hair pin isn't very consistent, so I'll need to train again later on and draw the pin myself for the training to be able to up scale it and actually make it look how it's meant to.
You can upscale the images to 550x550, and you will get an all NANs error if you try to go higher because there are no higher res images in the data set yet.
The hands leave a lot to be desired, but you can always use controlnet to make it better. I'll upload improved images next time when I update this.
EDIT: If you want a higher chance of getting her in different locations + full body, start with a portrait image of her, and use the outpainting technique demonstrated in this video.
Description
Trained on 28 images that were mostly portraits only
Cannot be upscaled but inpainting for the hair and face will fix it
Images that aren't portraits will only look Mamori-adjacent until you inpaint but when you inpaint it will fix it immediately
Hair pin lacks a lot to be desired
I will need to tag the backgrounds next time so that the backgrounds don't default to the same nature background
If you think about the hands, you will have a curse on your household. Please use controlnet to correct them for now.
FAQ
Comments (3)
I hope you will update it
I have learned many things since posting this first Lora.
1. Stable Diffusion changes the colors some without your consent. Darker or lighter than the original colors. I may be able to help this by making sure every single input of the hair color has it looking consistent, which will mean editing to color correct every single image in the data set.
2. The weird overlay on top of the images you see in the examples is because a picture from a flash back contaminated the entire data set and was baked in. I have removed them from the data set so they don't look this way next time.
3. In some animes, the characters get very off model. You have to make sure to correct things like missing eye highlights, extra eye highlights, pay attention to hair hair light counts and edit so they consistently only have that many, fix faces on things like Mamori's cat hair pin and the overall shape... Etc. Animes that manage to stay very on model have major respect from me.
4. Details in eyes will often be lost. It's hard to train it so it understands new eye concepts, even with larger, higher quality images. But the main reason these details become lost is due to the initial training data just being super flawed. When you make a Lora, you'll find it can often take low quality and boost the quality because it's looking at similar things in the data set to compare it to to utilize your new concept. So it takes from what already exists to go from there. So if the initial data set didn't have every single image heavily scrutinized, then that explains poor results.
5. If it has never seen a concept before, training it to do it is exceedingly difficult. For example, if a checkpoint was built from scratch with only pictures of men, you couldn't make a woman. This goes for obscure poses and hyper specific outfits. You have to do a lot to make the AI do what you need to in those situations.
6. Hands, feet, and genitals are never the same. That's why they don't turn out well often. Same for text. Text is very convoluted, too. You'd think it could understand the concept of letters arranged in a specific way, especially if you train on text for long enough, but no. It really struggles.
Not one model of Mirei Shikishima on this site? Every other female character gets a model but not Mirei? That's no fun.