IC LoRA

This is on Huggingface; I left it in the model's description.

https://huggingface.co/Alissonerdx/LTX-Best-Face-ID

Tell me what problem you encountered because so far I haven't encountered many problems; you probably don't know how to use it or you don't read the instructions.

@Dumcluck51 Try the second pass workflow that I left in HF.

@Udimuz If you're trying to generate full-body video, clearly the face will be worse, since this model was trained on close-up videos. Just read the description; there are some people on Banocodo who managed to combine I2V + Best Face ID and are getting incredible results that they would never have gotten with I2V alone, especially when the character's face isn't visible and they need to turn towards the camera.

@dav79mail156 Yes, in the second pass it does change sometimes, not always, since it introduces noise to achieve upscaling. I haven't really experimented enough to give you an answer right now. Regarding head swapping, I think it's possible to use it together, but I haven't tested it yet either.

Yes, totally possible, If I remove the audio part, the model will be half the size 1.2gb

If you didn't understand, then you must not have made it work correctly. If you use an I2V, you'll always generate something starting from that image, meaning you'll always have to generate the initial frame. Every camera change will always start from the image. If you use a LoRa identity, you can provide any prompt, and it will generate the video without starting from a first frame, since no first frame is being used, only a reference. What will make things happen is the prompt. Besides, you can use this to improve facial consistency since it's a LoRa IC.

If you look at my samples, you'll see the prompt format. To really use this model in a pleasant way, you have to pay attention to the prompt. It was trained on videos from a dataset that focuses on close-ups, so close-up videos will be better than full-body videos since there wasn't a full-body video in the dataset. Another thing you'll notice if you look at the samples is that the generated video doesn't start from the reference image, meaning I can customize everything with the prompt: change the clothes, change the scenery, change the action.

In scenarios where you want to generate a long video, you will always have to provide the keyframes, that is, generate the images beforehand. Using this model, you can simply generate continuously using only prompts. For example, music clips, etc.

That's because nodes involving v2v will always use more VRAM since you're providing an entire video as a reference, which will be much more resource-intensive than i2v or t2v.

You can do this for custom audio https://imgur.com/a/s2GuAvk

I need to update this workflow with this.

You can try applying and doing a second pass to try and solve some distortion problems, but as you already know, this is a LoRa and it won't solve a problem that has existed for a long time.

This is actually a technique that forces identity; it's not a conventionally trained IC lora , and that's why it requires my custom node to work. In the model's huggingface, I've included an article explaining how this works, but basically, it involves placing the reference in an offset space within the ROPE. This technique is called TASS-ROPE, and I needed to customize some code from the official training script to make it work.

@hboxgames132 Yes, it works. You could provide the first image and then add a latent to it over the empty latents.

@clzpetn804 Claude code hahaha and reading the arxiv articles.

@hboxgames132 I think TenStrip made a node for this; you have to join the Sulphur Discord server. https://discord.gg/Tqqmq9r2C

You can test this with i2v, v2v, but the training focus was ref2v, which is a IC LoRa , so it was designed to work with v2v. The issue that needs testing is that I have a custom node to make the reference work. I'll try to run some tests today and bring new workflows soon.