Video character replacement with no limit on video length, using Wan2.2 Animate Model.
It eliminates the phenomenon of gradually deteriorating image quality when generating long videos.
or try it out first to decide whether to install it. in case you are not satisfied.
Free 4090 online ComfyUI: https://www.runninghub.ai/post/1972295773822119937
Sign up this site to use free 4090 online ComfyUI, you'll receive 1000 RHCoins(worth $5); if not, a dialog will appear asking you to enter an invitation code — enter rh-v1213.
Description
FAQ
Comments (9)
This sounds really promising, because deteriorating image quality is something I fought with for many hours with vace to try and resolve, and even then when I got an 'unlimited length' chain workflow to work, the results I got could save the character quality but would fry the background some. It never really worked to my ideal. I'll definitely be checking this out and seeing how it goes. Thanks!
I tried a 15-second video with no problems due to my device; if you have more powerful equipment you can try longer videos. Thank you very much for your testing.
Can't install Segformer so the workflow is pretty much useless.
I installed segformer after some troubleshooting and got the workflow to work. Here's a note I made on it, in case you're running into the same issues I was:
First you install layer style node package through the standard comfyui installer, then MAKE SURE you have git-lfs package on your machine to be able to download 'l'arger 'f'ile 's'ystems. If on linux run just run your standard apt-get stuff. Then clone the https://huggingface.co/mattmdjaga/segformer_b2_clothes into your models directory. The node specifically looks for a folder called segformer_b2_clothes in your model directory. If you did not install git-lfs package and just directly clone it, and it didn't install everything, you'll get a 'header too large' error.
A heads up: LayerMask is a slow node, and it's not particularly clear it's running, because it doesn't output a 'starting' message, only a 'finished' message. I wonder if there is a faster node for this.... maybe not. I would have to check what I've tried previously.
I just got the workflow to work, so I'm still testing it, but I'll just edit this comment on thoughts afterwards or make a new comment.
A couple notes for anyone that is wanting to start up this workflow and having trouble.
1. It's not particularly clear from the civitai post alone BUT if you want the full workflow to work, there is a node blocked off on this civitai download and only available on their runninghub url. So the OP is making you go to their invite link to run stuff, or export the workflow from their link to bring it locally. It was free after a sign-in, but if that's a problem for you for whatever reason, then this workflow is not for you.
2. The LayerMask: Segformer node is a bit of a pain the ass to get to work. First you install layer mask node package through the standard comfyui installer, then MAKE SURE you have git-lfs package on your machine to be able to download 'l'arger 'f'ile 's'ystems. If on linux run just run your standard apt-get stuff. Then clone the https://huggingface.co/mattmdjaga/segformer_b2_clothes into your models directory. The node specifically looks for a folder called segformer_b2_clothes in your model directory. If you did not install git-lfs package and just directly clone it, and it didn't install everything, you'll get a 'header too large' error.
A heads up: LayerMask is a slow node, and it's not particularly clear it's running, because it doesn't output a 'starting' message, only a 'finished' message. I wonder if there is a faster node for this.... maybe not. Would have to check what I've tried previously.
3. If you get a tensor mismatch error on ksampler, you probably ignored OP's note of having your width/height in something in a multiple of 16, or you're trying to use incompatible models and stuff somewhere. I wonder if this w/h could be auto adjusted using some modulo or something somewhere, but that's a fight for another day.
Thank you very much for your careful evaluation. ComfyUI is indeed a great tool. I completely understand the frustration when users open an unfamiliar workflow and encounter many error pop-ups. In the ComfyUI environment there are so many nodes with similar or even identical functions. I tried to make my Comfy environment perfect, but frequent updates and errors defeated me. So rather than wasting time dealing with errors, I prefer to use that time to create value. That’s why I’m happy to recommend cloud platforms.
Oh yeah, no worries. It didn't bother me personally, because I mean, everyone gotta earn their bread, and sign up was free, not to mention you're providing this workflow for free too lol. I just wanted to point it out.
Alright, so I tinkered with this workflow some. I feel like I still have much to learn for Wan Animate, but here is my general review:
The workflow itself is great. I generated some 30 second long videos (H100) and it had no problem. It didn't have the usual amount of 'frying' of the video that standard continuation loops have and it's seamless. It still has artifacts but it's more consistent, if that makes sense. Maybe it's something to do with WanAnimate's capabilities specifically? I would have to investigate.
Some points of improvement I'd like to see may be limited by the current tech of WanAnimate I think but I'll state them anyways.
- I'm finding WanAnimate is a different beast compared to VACE. It does more simulation compared to VACE which is good and bad. It is good at exchanging the subject exactly just how the ref photo has it (something VACE struggles with sometimes) but it also can simulate a bit... too much. It likes to add unnecessary jiggle physics, for instance, so suddenly your person's thighs, breasts, calves - everything starts spasming at any movement unrealistically. Having a way to improve stability with controlling the physics simulations would be ideal if that even exists. This is the biggest gripe I have but also the hardest to address, because it's not just like a knob to turn.
- A smaller "nice to have" feature would auto adjustment for the number input so that it adjusts it to the nearest numbers divisible by 16.
- I'm getting some "lora key not loaded: blocks.9.ffn.0.lora_B.weight" I think on the lightning lora(?) which makes me wonder how much it is actually doing to help.
- I don't know if it would help necessarily but being able to load from a path would be pretty cool if it improves video size limitation, since the video upload in comfyui is very limited to the mb that can be uploaded. An even cooler (but way more challenging thing) would be a way to segment a super long video and reprocess it fully, but there may be limitations there too.
All in all, it's a solid workflow once you get the initial requirements set up. It makes me wonder if there is a use case where someone creates an I2V workflow chain (which will fry and warp the ever living shit of the subject after a certain number), and then subject replace the main person with the original image subject, thus "restoring" the person. Need experimentation.
Workflow contain harmful code. After installing nodes and restart comfyUI deleted itself from disk.