LTX 2.3
V1.1
No changes to the main flow
Added 3 example flows with example files and instructions
2 frame interpolation example
Img2vid with motion transfer example
remake video adding sync or sound example
All have their own json flow with everything set up for the task and instructions.
All have the needed files, video ref, image refs, audio files, etx.
This should show you how to use it for the most part.
1-3 stages
1. Basic image to video
2. First/last frame interpolation
3. Multi frame interpolation
4. Video motion transfer with guide frames
5. Video insertion as a mid video ref.
6. Video reproduction adding sound and sync
Audio
1. Custom voice ref to guide the generated voice
2. Custom audio can be synced
All options are optional, if no video or image ref is used it does regular text to video.
All options can be used together when needed.
Included the images i used for interpolation for you to test with. She is an ai girl.
Notes everywhere, read. Enjoy :)
Description
No changes to the main flow
Added 3 example flows with example files and instructions
2 frame interpolation example
Img2vid with motion transfer example
remake video adding sync or sound example
All have their own json flow with everything set up for the task and instructions.
All have the needed files, video ref, image refs, audio files, etx.
This should show you how to use it for the most part.
FAQ
Comments (7)
thanks alot
That looks very promising. Thank you! Cannot find hqtalk lora anywhere. Is it original file name in the workflow?
Sorry :)
hq i speak of is
https://huggingface.co/AviadDahan/LTX-2.3-ID-LoRA-CelebVHQ-3K
The other choice is this
https://huggingface.co/AviadDahan/LTX-2.3-ID-LoRA-TalkVid-3K
@sy0ww4bb1984 thank you so much. Whatever I try, I cannot make the sound work, though. It's just high pitch noise, if i remove the previewer from SamplerCustomAdvanced, Video Combine fails with error, that audio frames are not provided.
@comthumb The audio is probably the most confusing thing about this flow tbh. I gave a few different ways to use audio and did not properly explain what each did and how to combine or not combine them. I still have much to do with this flow to make it easier, i just threw it out here because the video copy was actually cool and i did not see others doing it.
I will try to explain the what happens to the audio
First with no options enabled (audio ref bypassed and custom audio bypassed) it uses an empty audio latent by default allowing normal LTX audio gen.
When you enable the audio ref group and give it a ref voice, using the HQtalk lora it will attempt to change the voice of the generation to the one provided.
When you enable custom audio this removes all need for LTX to do audio. It takes the audio you give it and uses it to form the video using its tone, speed, etc to drive motion and sync.
You cant use custom audio and audio ref together. They do different things.
Now when using custom audio you have a mask, this is set at 0 to not change the audio. Custom audio group has this mask, as well as 2 small groups between the samplers to stop/start editing the audio.
EDIT - also when using the audio ref and HQ lora you need a specific prompt style i think i included and example
What are you trying to accomplish? I could probably explain better if i knew what you were trying to do. Easier than just telling you every possibility,
@sy0ww4bb1984 Thank you very much for the explanation about audio. I think there was compatibility issues with comfyui, as LTX's own sample workflows did not produce correct audio for me, but latest update fixed the problem. So far your workflow is the best I've tried. Great job.
@comthumb Thx for the comment tbh. None of my workflows are currently counting or showing any stats so i didnt really know if anyone is using any of them. I cant see likes, downloads or anything for over a week now.