For each model, see the version details for more information.
All the models here are experimental; they often won't work perfectly, the prompt will often need improvement, the input will often need alteration, or the parameters will need adjusting. Therefore, don't treat these models as something that should work perfectly right away for everything. They will only cease to be experimental when we discover a better way to do this.
Description
This model was trained on 8,000 video pairs, and training is still ongoing for a few thousand more steps. It is still experimental, not trained with a fully professional production target, and the model may be updated unexpectedly as new checkpoints.
The current goal is not final polished production quality, but to explore:
edit-anything behavior
prompt-following
inference tradeoffs
synthetic dataset building, especially for style data
The model was trained around four main prompt patterns:
AddAdd a/an [subject/object] with [clear visual attributes], [precise location in the scene].
RemoveRemove the [subject/object] [location or identifying description].
ReplaceReplace the [original subject/object] [location] with a/an [new subject/object] with [clear visual attributes].
Convert / StyleConvert the video into a [style name] style.
Workflow URL: https://huggingface.co/Alissonerdx/LTX-LoRAs/blob/main/workflows/ltx23_edit_anything_v1.1.json
One important thing during inference is CFG.
A good starting point is testing a distilled setup with CFG = 1. If the edit feels too weak or the model is not following the prompt well enough, increasing CFG can be the key. In some cases, increasing the LoRA strength to around 1.2 can also help.
The workflow is also not fully optimized yet. It still needs more testing to find the best combination of:
CFG
LoRA strength
number of steps
model combinations
It may also be interesting to combine this model with other models and see what kinds of results emerge.
If you can test it, please share your findings. Feedback on prompt behavior, edit strength, consistency, style transfer, and failure cases would be very helpful while training is still in progress.
Another very important thing is that the Removal task should have a very clear direction indicating where you want to remove what you want to remove.
Examples:
Remove the black robot sitting at the table.
Remove the person riding the electric scooter on the left.
Remove the person with glasses and the microphone in the foreground.
Remove the image of the green trees on the top left.
Remove the woman and the smoking bottle.
For example, if the object are in front, use foreground ... background, left, right, top, bottom.
Another way to remove things that don't want to be removed is to simply add a mask, for example magenta, over the object you want to remove, and use this video as a guide. When writing the prompt, you write something like: "Remove object masked with the pink color." Sometimes this is much more precise than waiting for it to recognize what actually needs to be removed, because in this case the biggest indicator is the magenta-colored object.
FAQ
Comments (81)
Straight to my Hidden Gems collection!
This is definitely one of the best LTX LoRA ever published. Thank you for sharing 🙌
This is amazing!! You are a legend now!!!
Edit: Damn... You have opened the doors of hell with this one also.
perfect
Really interesting that it can target a part of the video without needing to use SAM. In my testing, it has been a bit hit and miss but still very impressive. I'll be keeping an eye out for any updates, for sure.
Download the new version and test it again, and tell me if anything has improved. I just uploaded a 9000-step version.
For those who have already downloaded it, I just published a new version in step 9000. Test this one; it seems a bit better in my opinion. Also, I updated the workflow to use the new Lora distillation instead of the one extracted by Kijai. And if you can, of course, share your tests here; I'll reward you with buzz.
let me cook
dude this is god-tier shit! nice.
also, just fyi, i believe you can release updated step versions as like v1.1 or whatever so people are able to compare step versions for themself? just would be cool for historical sake as well :D
Very interesting, I wonder if this can somehow work with Wangp. Edit: just saw this on reddit "This is amazing.
For WAN2GP users:: It works by loading the LoRA and using the video as TX2 Raw Format / Control Video for Ic Lora with Control Video Strength (higher = closer to the Control Video) = 1, you can also activate Generate Video based on Control Video + its Audio Track and Text Prompt."
I have also been told that skipping phase 2 and ONLY using phase 1 (something now possible in wan2gp) works best. Phase 2 ruins consistency?
I tried to make it work in drawthings for mac but no luck. Maybe i'm just not understanding it. Well done anyhow this will change the game for LTX!
Just when i was about to give up with LTX 2.3. You release this magic
Did you try the new distilled 1.1 version of the model that just came out a few days ago? (Kijai has it as well as QuantStack GGUF's). It honestly is a VERY surprising leap in prompt adherence and scene coherence (WAY less mutations and it feels similar to how WAN 2.2 functions in regards to prompts that are not verbose or properly formatted).
This works shockingly good already, holy shit. You cooked bro!
It works well~ Thank you. I told it didn't work but it was a problem of ltx lora master loader I added.
Is there any workflow that could allow adding objects using reference images?
That would take this to the next level.
I have one called Masked R2V, I saw that someone made a video about it today.
https://www.youtube.com/watch?v=lQcter3uVQA
But I'm adjusting the dataset to do this: a new training with references.
@NRDX cool. Also, someone has noted the style change cant quite do Pixar 3D. Not sure if adding that to your dataset would help or not. Perhaps the style changes but not quite like morphing them into a true Pixar character exactly.
@NRDX if possible to improve its understanding of different necklace types, breast styles, and penises.
@kronos1959777 You can try using a style lora in conjunction and try using a trigger from it without altering the style.
@kronos1959777 Have you tried doing the morph with something like Flux Klein Edit, then doing a first/last frame LTX?
@sonofabean well for this lora i am thinking of it just adding an input reference image and have it replace or add the object you want with that without needing image edits or first and last frame.
@kronos1959777 I'm currently working on this, hoping to have a v2 or an alternative v1 with ref support by tomorrow or the day after tomorrow.
what would be the best way to use this on a video that is say 40 seconds long instead of 5 or 10 and have it keep consistency?
Context Window
@NRDX Wish I could understand that pull request. @snake88 lemme know if you find a way to do it please
is there a good minimum resolution? I tried this on reduced resolution ~200x500 and it resulted in the replaced item being much larger than the original...
oh I think I see the issue maybe has to do with trying to edit a very long video..
this is just AMAZING... works surprisingly well!! Nice job, thanks!
very nice, thanks! Anybody managed to get other LORAs working in combination?
Used the WF mentioned in the description, was able to add lora just like any other WFs. Working for me
A guy on Reddit claims to have succeeded, but I haven't tried.
https://www.reddit.com/r/StableDiffusion/comments/1sp03jq/comment/ogz5yn1/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button
Once I get to the sampler, I get this error:
"RuntimeError: The size of tensor a (25600) must match the size of tensor b (3293056) at non-singleton dimension 2"
Tried different models, distilled loras or not, other versions of the models.
Just WOW! You are awesome, dude. This is great LORA
We will soon have a version that supports referencing. This one is currently in the training phase.
we will let you cook
An update: I'm currently trying to create a method where it's possible to pass the reference image in a way that isn't in the first frame of the video guide. I actually trained that way, but it's not very good, besides the fact that there isn't a high-quality dataset with v2v + reference images, so I'm having to build one. This takes a while because it's very complicated to produce datasets of this type as it involves processing through other existing methods.
@NRDX Mind sharing your method and approach so maybe people could collaborate and provide/create training data?
@fenasikerim Right now, we do not have an established way to provide reference images with LTX-2.3. You can see this kind of capability in several Wan-based releases, such as VACE, BindWeave, Kiwi, Ditto, and others, but for LTX-2.3 we still do not have anything at that level.
At the moment, the only practical way I have to do something similar is through IC LoRA training, where I can build a template around the guide video. Just for context, to train a model like EditAnything, we need paired data: the guide video and the target video. The guide is the original video, and the target is the edited version. On top of that, we also need a prompt to describe the transformation from guide to target.
So in this setup, we already have two video inputs, but we do not have a native way to inject an external image as an additional conditioning signal unless we customize the training scripts.
What I usually do is rely on one of two approaches.
The first approach is to place the reference image as the first frame of the guide video. This means that, in the first frame, the model sees the object I want it to use later inside the scene shown in the following frames. This is the approach I am trying right now. The problem is that this reference is a very weak signal compared to the rest of the video, because it is only a single frame. The model can simply ignore it. Even if it does pay attention to that frame, it may still stop using the reference when something changes in the video, such as a camera cut, camera movement, or when an object appears or disappears from the scene. In practice, the model can become lazy and just reconstruct the original video content instead of applying the intended edit.
The second approach, which I know works much better, is to create a custom guide video template. This is similar to some of the reference-based inpainting models I posted before, or my head-swap setup. In those cases, the guide video contains a green chroma-key region, and the reference image is placed inside that region. This way, the object information is present in every frame of the guide video. The model always has a visible reference to copy from, which solves the problem of the model ignoring it. The downside is that many people do not like this more customized template format. Still, if I cannot find a better solution, I will probably move forward with this custom-template technique.
Dataset creation is another major challenge. Besides the guide video, which is just the original video and is usually the easiest part to obtain, I also need the target video, where the object has already been inserted, and I need a clean image of that inserted object to place into the guide template. That object image is usually the hardest part. Most available datasets do not include it. And when they do, they often extract it directly from the target video itself. That is not ideal, because the extracted image ends up looking too similar to the target video frame. The correct setup would be to use a clean standalone image, not something extracted from the video, since extraction can introduce distortion or leave pieces of the original background visible. If you train the model that way, the quality can collapse and the model can turn out terrible.
So that is basically where I am right now. Training reference-based models is extremely difficult, both because the conditioning mechanism is weak in current LTX-2.3 workflows and because building a proper dataset for this kind of task is very hard.
where find working workflow for v2v? all i tried return runtime.
it isn't working for me.
The video quality is extremely low, background replaced by a coarse grid of large, distinct, multicolored square pixels. The overall resolution appears severely degraded.
human form is reduced to a vague, ghost-like silhouette. details of person are completely unrecognizable, swallowed by the digital noise.
i'm using ltx-2.3-22b-dev_transformer_only_fp8_scaled this with distilled lora.
and kijai video audio vae.
I also tried "ltx-2.3-22b-distilled-1.1_transformer_only_fp8_scaled" without distilled lora. same result.
what am i doing wrong?
It would be easier to see a screenshot of the settings and also understand its purpose; not everything is possible with LoRa. We need to remember that this is LoRa, hahaha, it was made to work with objects and not with gigantic and extravagant things.
@NRDX I downloaded the wf from reddit which isn't working. now I tried that wf with wonder woman preview and the weird artifacts are all gone but it doesn't edit anything like adding a cat in the background.
been trying for 5hrs. literally using the exact same workflow in the post, still not working with the same models. gg
@fluxxes You need to use the workflow that's in HuggingFace.
Works very well! Still, one question: I noticed that when changing something around the neckline or the hair of a person, that the face completely changes as well. Were things trained using bounding boxes? perhaps using more detailed masks gives better results?
Things weren't trained with bounding boxes; there's no masking in this model's training. I think it depends a lot on how you write your prompt and what command you're going to use. The prompt here is 100% what dictates everything, so it needs to be well-done. Besides that, some things like increasing the LoRa strength can sometimes create distortions. Try different strengths, different CFGs too, and focus on the prompt and the instructions I left about it.
@NRDX Does it then also respond to "things to NOT change"?
@wxcvbnw I haven't actually tested this, but you could try using masks along with the mask model on the area you want to change. I haven't tried this either, but it might work.
@NRDX I truly tried masking but i'm struggling to get things working (i'm not good with making workflows and it errors). I currently do 🅛🅣🅧 LTXV Preprocess Masks to 🅛🅣🅧 LTXV Set Video Latent Noise Masks into the LTXVConcatAVLatent? Perhaps if you know how to do it maybe add it to your next workflow :).
Amazing lora! Can something like this be created for Wan 2.2, as well?
@NRDX will this be possible? Cuz Wan 2.2 has richer lora ecosystem, so a lora like this would be really useful to mix n match with other existing loras. Please do consider. Would love to support it, if needed :)
@honryindian For Wan, there are already many models that do the same thing, Ditto, Kiwi, VACE...
@NRDX Ditto is dead, they haven't released the local model. Kiwi is more of a research project and never got any good workflow to make use of vace. I'll gladly switch to LTX, just to use this LoRA. Just need to find a way to get the WF working on longer videos
@honryindian it is possible to make long videos by taking the last frame and putting it through klein9b, just vae-encode it and use latent as reference for the conditioning nodes in any edit WF and also use latent in ksampler at 50% denoise at 4 steps instead of 8 using distilled model. use eular + simple, if you do more than 4 steps with ksampler it tends to ruin the image. Prompt: Do not change the composition, only improve the image quality. And then it should be very seamless between clips, the only problem is drift as characters tend to change over time with LTX, oh and LTX isn't as good as wan when following prompts you have to be very specific.
Just swap out the custom sampler in the typical edit mode WF with a ksampler to be able to use denoise setting and drop the steps to 4 from 8. Bonus the ksampler at 4 steps is faster and generally gives the same quality.
don't known why output video is empty (black screen)
I discovered an issue: if the character in the original video is in an intense state of motion, once I remove their jacket, the character's movements become smaller or even stop. I wonder if anyone else has encountered this situation.
Did you try doubling the frame rate for the generation? The OP shows some tests he did with fast motion
Is it possible to edit it so it can take an input image as a reference, for the replaced subject, etc., etc.? I have already made an edit to the workflow to include an LTX2 Sampling Preview Override node so that the generated video, while sampling, can be checked.
So I wondered if reference image input is possible?
i believe that is what he is working on next, he posted in the comments about it a few days ago :)
@jackaroo1432111 Thank you very much for letting me know.
Please sir make a lora for ltx 2.3 i2V fml in which the pupil of the eye is clearly visible. please please.... This is a very big problem
I tried masking an object using sam2 or something I believe, just a regular black and white mask, in wan2gp, and it couldn't remove the object. Do I need to maybe try remove the white masked object or something specific? I tried remove the masked object.
Try one phase. It's work well.
Mask + one phase
@oatkkth728 send a workflow link if you would be so kind
try magenta color
I tried just about every combination I could think of to remove objects using mask. So far only using Whole Frame works, at least in wan2gp for me. I tried various masking and prompts. Anyone have a solution? This sometimes struggles to remove objects.
Does increasing the lora strength generally help with all this lora can do?
yeah, mask + one phase does it perfectly
@luckyblue82 if you would be so kind so share a workflow kind sir
Wan2GP instructions:
Control Video: Ltx2 Raw Format/Control video for ic Lora
Generate Video baased on Control Video+its Audio Track and Test Prompt
Advanced Mode->General->Phases->One Phase
You can do whole area or masking, but I did not find masking to be necessary once I switched to One Phase
For me, I have found--very strangely--that some aspect ratios or resolutions work better than others. Even if my reference video is 3840x2176, I cannot output an edited video with any clear or good changes. But the same video with 2880x1600 as the output will work PERFECTLY.
Hi Kronos you seem to be playing a lot with this. Have you found a working masking setup? or what other interesting findings did you have? Maybe if so you could share your workflow?
Very cool - could we get a Sulphur Edition?
NSFW no haha
Thank you for this! It´s such a fun and cool project.
The workflow url no longer works.
Details
Files
ltx23_edit_anything_global_rank128_v1_6000steps_prodigy.safetensors
Mirrors
ltx23_edit_anything_global_rank128_v1_6000steps_prodigy.safetensors
ltx23_edit_anything_global_rank128_v1_6000steps_prodigy.safetensors
ltx23_edit_anything_global_rank128_v1_6000steps_prodigy.safetensors
ltx23_edit_anything_global_rank128_v1_6000steps_prodigy.safetensors
ltx23_edit_anything_global_rank128_v1_6000steps_prodigy.safetensors
ltx23_edit_anything_global_rank128_v1_6000steps_prodigy.safetensors
ltx23_edit_anything_global_rank128_v1_6000steps_prodigy.safetensors
ltx23_edit_anything_global_rank128_v1_9000steps_adamw.safetensors
Mirrors
ltx23_edit_anything_global_rank128_v1_9000steps_adamw.safetensors
ltx23_edit_anything_global_rank128_v1_9000steps_adamw.safetensors
ltx23_edit_anything_global_rank128_v1_9000steps_adamw.safetensors
ltx23_edit_anything_global_rank128_v1_9000steps_adamw.safetensors
LTX23_Edit-Anything_v1.safetensors
ltx23_edit_anything_global_rank128_v1_9000steps_adamw.safetensors
ltx23_edit_anything_global_rank128_v1_9000steps_adamw.safetensors
ltx23_edit_anything_global_rank128_v1_9000steps_adamw.safetensors
ltx23_edit_anything_global_rank128_v1_9000steps_adamw.safetensors
ltx23_edit_anything_global_rank128_v1_9000steps_adamw.safetensors
ltx23_edit_anything_global_rank128_v1_9000steps_adamw.safetensors
LTX2_EditAnything.safetensors
ltx23_edit_anything_global_rank128_v1_9000steps_adamw.safetensors
LTX2_EditAnything.safetensors
ltx23_edit_anything_global_rank128_v1_9000steps_adamw-mid_2553102-vid_2869279.safetensors
ltx23_edit_anything_global_rank128_v1_9000steps_adamw.safetensors
ltx23_edit_anything_global_rank128_v1_9000steps_adamw-mid_2553102-vid_2869279.safetensors