This LoRA is trained on in-game art and fanart from the Hades and Hades II. This model is trained on Pony Diffusion V6 XL.
In the second version, we've added more images of many minor characters and included a cosplay version of each character to achieve more realistic results. This enhancement ensures that even the lesser-known figures from the game are represented with greater detail and authenticity, while the cosplay versions add a layer of realism that brings these characters to life in new and exciting ways.
Update:
03/26/2025 – I’ve added a version trained on Illustrious as the base model, using the same dataset as Pony V2.2. This model is compatible with Illustrious 1.0, 0.1, and other Illustrious-based checkpoints. It provides an alternative option, but overall, I feel the Pony version is better.
03/24/2025 – I’ve added Version 2.2, a model trained on an updated dataset that includes characters from the Hades: Warsong update. This version also removes non-aligned style images (including cosplay images) from the dataset for better consistency.
Usage Tips:
Use with Pony Diffusion or a Pony-based model.
Use with a pony prompt.
Trigger words for character (some name might not work with old versions):
Female:
melinoe,aphrodite,arthemis,athena,alecto, arachne,demeter,dora,dusa,eris,eurydice,hecate,hera,hestia, medea,megaera,nemesis,nyx,orpheus,persephone,scylla,selene,tisiphone,male:
zagreus,achilles,apollo,ares,asterius, chaos,charon,chronos,dionysus,hades,hephaestus,heracles,hermes,hypnos, icarus,moros,odysseus,patroclus,polyhemus,poseidon,prometheus,sisyphus,skelly, thanatos,theseus,zeus,
You can also add the lora for mimicking the Hades style
Use a trigger word
photorealisticto get cosplay version of a character. Since Pony didn't train on photorealistic image, you better remove tags likescore_9,score_8_up,score_7_up, ... or use them in negative prompt.
Description
Add more image to some minor characters.
and add cosplay of characters with the tag 'photorealistic'
FAQ
Comments (17)
wow, your loras are very impressive, could you please make an article describing your process,plese? or just awser a question how many steps your loras take usually?
My process is improving with each LoRA I create. It's not perfect yet, so I don't feel ready to write an article at this time. However, I am happy to answer any questions you may have. Here’s an overview of my approach, where I use ChatGPT to automate my pipeline:
Step 1: Dataset Scraping
I typically create my dataset from high-definition videos, selecting frames and using YoloWorld (for cartoons) or YoloV8 (for real people) to crop the images. For the Hades LoRA, I tried a different method: I used WaifuC to search and download data, and PreSize to crop the images.
Step 2: Labeling
This step still requires human input, which can be a bit tedious. I tag every image by the character's name and style. In the first version of the Hades LoRA, I used BooruDatasetTagManager. However, I found it easier to drop images into folders named after the characters, so I continue to use this method.
Step 3: Prompting
In this step, I use popular tools like WD14 and BLIP to tag each image. With my pipeline, I randomly select prompts from WD14 or BLIP and add the character's name.
Step 4: Training
I use Kohya_ss for training. I follow the provided presets and train the model until I am satisfied with the results (or until I am eager to start a new project).
Step 5: Post-Processing
I use A1111 to review the results. This step often involves multiple checkpoints, which I combine using the SuperMerger plugin in A1111. By plotting X/Y/Z, I compare the results and ratios before combining them.
@titansteng Thats very interesting, thanks for your response.
@OliverDR CivitAI article are full with great resources. I still learn from them and improve my pipeline every model.
@titansteng Interesting, I still think that training data you have filled including poor quality blurred screenshots (perhaps in an attempt to make better integrity?) Although cropping has become much more accurate and focus, though because of this image resolution has also decreased, I prefer upscale, but many say that above 1024 does not make sense. So I wanted to ask at what point does gpt take part in this? Automation of something? Or just an advisor?
@titansteng I also wanted to ask if you solve the loss problem somehow? I mean let's say heydes has almost 3000 images, supposedly train batch size should be at least 2, and preferably maybe 4 or more. Or don't you think about it? I've only heard about it, alas I don't have enough equipment to test what the experts say. (Besides most of the time their articles are not directly about pony for example, not always even stable defussion) Also I'm still confused by your caption, it's interesting but also weird, I mean, basic lora use wd14 and put in what they want to unbind, in the case of a character let's say hair, eye color, maybe breast size if desired. (Change is possible and just so, but in some cases if you ask instead of red eyes yellow, will be more like a mixture of reddish yellow) pony is very flexible, but nevertheless signatures like score_ use as far as I know only when training caption models checkpoints, for yes indicating the category. Blip usually in realistic models or where it is impossible to describe with tags (blip is more clear and more...). cohesive? In general more cohesive than a set of tokens describing the same thing in chunks). Have you tested the "standard variants" and compared the "score" Variants to them? Although it seems you are using this to make the photorealistic tag work? If the model understands what score is, then don't the other score tags still work? I mean, Score is made in a very peculiar way, it's more about the type of content and its quality than about the aesthetics of each image like animagine. My max lora has just under 3k images and I'm not comfortable with it due to epoc being divided into 3 thousand steps and then immediately 6 thousand. (Although of course you can try to get around it a bit with lr within reason) Also I am confused about the digestibility, lora does not seem to have been made for e.g. 10 thousand images. In case of for example dreambooth I am more sure that the model reads the image (as far as I understand animagine was made at one stage through it and there seems to be more than several hundred thousand images) besides the signature approach is different, but in case of lora your approach is interesting and different from the standard one. (If you want you can write me in private messages, I just thought I'm not the only one interested in this and replying here if they are not different will allow others to see your information.
@happyhen My new dataset pipeline already includes an upscale step to a resolution of 1024. For instance, in my new Hades Lora v2.1, I use aiarty for upscaling. Initially, I used stable diffusion to upscale all the images in the dataset. However, this step was very time-consuming, so I removed it from my pipeline. I now address low-resolution artifacts using the SuperMerger plugin. I merge my model with a shape model of similar style, focusing on only the first and last 8 layers. However, this step requires human intervention to review the results and select an appropriate ratio. On the other hands, while aiarty works well for anime and cartoons, it struggles with realistic images, especially with issues like motion blurring. I'm still searching for a better option to automate in my dataset preparation pipeline.
@happyhen I use ChatGPT to create scripts for automating my dataset processing. For example, it has helped me write a program to turn video clips into a list of image folders, separated by class and already cropped. In the future, I can also use ChatGPT to include upscaling step to my pipeline, if I find good upscaling AI.
@happyhen About the batch size, I use as much as my machine can handle. For this Hades Lora, I use a batch size of 12. I don't have enough experience to say much about this point, but I think a batch size of 2-4 could also work if your machine is not powerful.
@happyhen Similar to what you said, my first version of Hades Lora used only wd14 and the character name as image caption. At that time, I used BooruDatasetTagManager to add the character name (in case there was more than one character in an image) and fix anything wrong from wd14 auto caption. However, users complained that my model didn't remember the character well if they didn't use the wd14 format. For example, Aphrodite required "pink hair" or "long hair" in the prompt besides her name. I believe some users also don't like to prompt in the wd14 format. So, to solve this problem, I use both blip caption and wd14 caption randomly in my later versions. My Lora model can handle both prompting styles. Interestingly, the Pony Diffusion model also doesn't use wd14 captions.
@happyhen In the future, I also want to include LLM caption like mention in Pony V7 and LEOSAM. It is very promising. But right now, I can't find any cost effetely way for me to use it.
@happyhen I appreciate your questions and would like to answer them, but I'm a bit confused by some parts. Could you help clarify each question for me (question that still not be answered)?
@titansteng sounds cool, if the video would automatically find the right character and trimmed, as well as sent to different folders by quality, it would certainly be incredibly useful, as far as I understand you have it easier, but one day we will see something similar. In the case of blip considering how the author is trying to get away with "live language" I'm sure we will get an even better level of implementation with pony 6.9. Not quite sure what you are doing with supermerger, going to merge what models? Different lora? Thanks for the replies.
@happyhen SuperMerger can have many different use cases. The idea is that the core layer (middle) controls the overall appearance of the output, like character identity, while the fine layer (first and last layers) controls the style, including the blurriness of the result. Many people use it to mix different layers of different checkpoints to create new styles. However, in this context, I mention it because it can be used to polish our Lora model if it overfits the low-quality blurriness from the dataset. We can mix the fine layer of our Lora model with another Lora model with a similar style but without the blurriness. You can experiment with different layers and different ratios. Personally, I stick to changing only the first 8 layers and the last 8 layers.
@happyhen Also, I believe a reason the wd14 style prompt is so popular is because the clip model in sd1.5 and sdxl is not powerful enough to understand long sentence prompting. However, the upcoming wave of base models (whether pixart, luminar, or sd3) use better text understanding models. They seem to have a better understanding of sentences. So, I'm interested in using long sentence captions like LLM captioning.
@titansteng so what if i extract a lora from the base model and merge the first and last layers it would become a "style free" lora? I did some merging with a extracted lora of pony, and i did with a few lora of sd1.5 but it used a anime lora, to some loras to have a better result in the genereted images, but i dont know the way it works tihs depth.
@OliverDR I guess the style will depend on the base model that you apply the merged lora with. Or you can get a lot of artifacts if your lora is too different from the base model that you are using.