Training Multi Character Loras for Flux - First Attempts

Welcome,

With 2k cost minimum of training flux lora quite a few users can actually do that regularly and training single character unless it's "top tier waifu" won't even pay for itself.

So i keep tying to make on-sitre traing for multi character loras work, with this article i want to share what i realized and invite You to share Your insight.

My first attempt was to make pack of Dungeon Meshi main crew, then i had completely no idea what i'm doing, now i'm just kinda have no idea what i'm doing.

The process was: default settings for flux lora training, captions, i changed nothing in training parameters.

It was completely failure, it got my unholy amalgamations of the group features, unable to generate any of them.

Then i watched Steams with KFTiger, and read Valstrix's Crash-Course Guide to LoRA (& LyCORIS) Training, after that i realized two things:

• Not every Flux lora need captions, you can train character/style with tags, and keep captions for more advanced loras.

• The most important change in training setting is to put train batch size at 1

With that insight i threw training data from Pony intro Flux to see will that work, and it did.

Here is my first sucesfull one: https://civitai.com/models/880035/relict-monsters-or-witcher-3?modelVersionId=994518

What i get out is that is that on site training can for sure handle 3 characters loras, all monsters had 25-30 images and got tagged.

Another part of experiment: Doing the same with more character, tried to combine all my Dandadan pony loras ( https://civitai.com/models/864530/dandadan-anime-characters-collection ) intro 1 flux lora, the same process as with the monsters, this time it failed, either too many character or tags performed worse.

So after buying gold i came to experimenting with stuff like that: run 3 characters from Dandadan.

Made version with tags and one with captions: https://civitai.com/models/940446/dandadan-flux-character-pack-part-1
Both versions are uploaded, at the first glance captions seemed failed/worse.

But with my initial testing actually Caption version seem to perform better this time.

I have over 44k buzz left and with gold and 100+ loras account i can for sure use some of it to keep testing how to make multi characters loras work consistently.

I will for sure keep testing tags vs captions and also experiment with improving tags/captions itself while training.

If You see something wrong with that process, or have already successful workflow for multi character loras let me know so we can learn the process to do it together.