Huggingface: https://huggingface.co/sheldonxxxx/MechaDream
Support me on Ko-fi to help me buy more GPU time: https://ko-fi.com/techadventurex
MechaDream
A Stable Diffusion model for Mecha
All in one model (ckpt+lora+vae in one safetensors): https://civarchive.com/models/47135
Recommended Setting
Checkpoint: Counterfeit-V2.5
VAE: mecha_v2_e3-pruned.ckpt (https://huggingface.co/sheldonxxxx/MechaDream/resolve/main/mecha_v2_e3-pruned.ckpt)
Sampler: UniPC (Personal preference)
CFG Scale: 15
Negative Embeddings: EasyNegative, bad-hands-5
Lora Weight: 0.8
Hires.fix with scale >= 2 is highly recommend
My workflow
Step 1: Batch generate images
Step 2: Select the best image and inpaint any flaws
Step 3: Upscale using MultiDiffusion
🡳 Some comparison images between original image and after the above workflow 🡳
Roadmap
MechaDream v1 will be the base model for generic mecha image generation
New Lora models train on different anime Blu-ray will add flavors into my MechaDream
List of WIP anime:
Gurren Lagann (Dataset is ready, will start training when I'm less busy)
Code Geass
Macross Frontier
Evangelion
Genesis of Aquarion (Maybe)
Valvrave
Eureka Seven (Maybe)
Gundam
Armored Core
Description
Base model: Counterfeit-V2.5
The Road to this Lora
Step 1: Fine-tuned on a dataset consisting of 130,000+ images of mecha/robots from various sources
Step 2: Mix the Lora models trained using different parameters
Step 3: Test different mixing ratios to find the best model
FAQ
Comments (6)
this is a really cool idea
since your dataset is that huge is there a way to control the lora better? Like if I wanted a specific mecha design aesthetic from ZoneOfTheEnders, CodeGeass, Front Mission, Armored Core, BreakBlade, etc
or is it all based on mostly Gundam?
This model is very biased toward Gundam, because this kind of anime series tag can only be obtain from booru site, but majority of mecha related images on those site are Gundam (at least from the set of search keyword I'm using).
My current plan is to train individual lora using screenshots from Bluray DVD, stay tuned for more model release 😉
@sxxxx oh yeah that makes sense, danbooru is filled with gundam when it comes mecha. I assume more "rare" franchises require individual image collection and screecaps/artbooks.
It's really cool though, if it aligns with your interests pls do one for the Escaflowne mechs that are more along the lines of giant knights/ancient tech⚔🤣
keep it up!
What about mechas from video games such as Armored Core, Zone of the Enders, Daemon X Machina, or Titanfall? They could be good additions to your lora.
I have it on my plan, will do it one by one, already running out of disk space ...
Do you think it's possible if you could add in Infinite Stratos images to train your Lora? The problem with Infinite Stratos is that the mechas are exoskeletal suits rather than giant pilotable mechas similar to gundams. The mech designs in Infinite Stratos hardly stands out as much as with other anime mechas. But it will be great if Stable Diffusion could design the IS units from the anime as giant mechas.