This is just the same dataset but trained for LTX 2.3 but I increased rank from 32 to 64 (thus the 1.2GB file size). The result is really great. We get 6 character likeness and voices which sound and look great. For one lora that's a lot of information! And the characters don't bleed into eachother.
Edit: All issues have been fixed in v1.1. Give it a try!
Add "subtitle text" to negatives if you get them, you should not get them otherwise.
Robert, Invisigal, Blonde Blazer (and without powers too), Prism, Malevola, Chase
Punchup is trained but not enough data, haven't tested. (char_punchup)
Royd is in the data as char_roy but also not much data.
Other characters are in the dataset but not tagged. Shroud is not in the data at all.v1.0
Style Trigger:
DISPSTYLE
Invisigal:
char_invisi, a woman with short dark hair accented by a purple streak, wearing a purple jacket over a dark top and distressed black jeans
Blonde Blazer:
char_bb is a woman with long blonde hair and a blue mask, wearing a blue and yellow superhero suit with a yellow cape and a red diamond-shaped gem on her chest.
(no costume, powerless)
char_bb has long, wavy dark brown hair and wears a strapless evening dress with long dark blue opera gloves
Robert Robertson:
char_rr has short brown hair and wears a light blue button-down shirt with the sleeves rolled up to his elbows and dark trousers.
Prism:
char_prism, She is a black woman with straight hair split down the middle, teal on the viewer's left and magenta on the right. She wears large, rectangular, reflective teal visors, blue lipstick, and a gold ring necklace. Her attire consists of a black sleeveless top, thick gold bands on her upper arms, a long magenta glove on her left arm, and a teal glove on her right hand
Malevola:
char_malev a woman with red skin, long black horns, and glowing yellow eyes, a long red tail,, is dressed in a white tank top and denim shorts. she has a sword attached to her back.
Chase:
char_chase, an older Black man with white hair in locs and a mustache, wears a yellow sweater over a light blue collared shirt and black trousers with a gold buckle belt. A pair of glasses hangs from the collar of his sweater.
Description
Fixed the issue with the dataset causing visi and bb's voice to merge. I added in an additional 100 or so missing video clips of visi and around 30 more of bb. Also I fixed some bad captions. Then I resumed from 24.5k steps and trained to 31k steps.
The result is much better voices for the two. I missed a few captions on rr's beard so if you see stubble on female characters just put "stubble" in the negative prompt.
Also roy and punchup's voice and likeness is slightly better now. But still not stable.
Overall very good result. I'm really proud of this one.
FAQ
Comments (11)
Fantastic work man, was this trained on musubi trainer the same as before?
also are you now doing it with LTX 2.3 checkpoints?
really would love your training data <3 dataset.toml + training_args.toml
either im always asking too much or im doing something wrong for my loras.
Thanks for the revision and adding in the training data! Gonna have fun with this and cross my fingers for Coupe inclusion some day, thanks again.
FYI I added some sample captions and videos, plus my settings files in the dataset download zip file.
Hi! How to make original characters but with this style?
The LORA system is extremely impressive. It enables six different roles to interact within a single LORA system while maintaining consistency. This has shown us the potential of LTX2.3. Additionally, I would like to ask you a question. Could you please tell me about the labels of the video training set and the tools you used to complete it? If you can answer, I would be very grateful!
I made a detailed write up here
https://www.reddit.com/r/StableDiffusion/comments/1rv40xc/showing_real_capability_of_ltx_loras_dispatch_ltx/
@tazmannner379 Thank you. I will read and study your tutorial!
I think I might do a v2 of this. Try adding more characters and fix the audio quality issue. I think a lower LR on the audio might fix it. Not sure when I'll try, my next lora may a different show
Almost done with a new coming lora, but from training it I learned putting the LR at half speed of the video LR makes the audio ok even at 35K steps. So, next version will have more characters and fixed audio issue.
Works pretty great in a V2V Workflow from a WAN 2.2 input.
Just saw your generation, really well done!