SVI Extend
https://github.com/vita-epfl/Stable-Video-Infinity/tree/svi_wan22
Create videos and extend them seemlessly using SVI.
Following SVI LoRAs are mandatory:
switch between default behaviour, anchor_samples and end_frames within the same subgraphs
connect an image to a part and enable the respective toggles to use end_frames or anchor_samples
NEW! v3
Extend existing videos using https://github.com/wallen0322/ComfyUI-Wan22FMLF
enable "video extension" toggle inside the settings
uses source video resolution by default
rescale video using the megapixel slider by enabling "video rescale" toggle
use included version of the nodes from inside .zip or download the latest version straight from the git if issues arise
More info inside the workflow.
AIO i2v+t2v
All in One workflow for for basic WAN 2.2 video generation.
Following features included:
Switch seemlessly between 2 and 3 sampler solutions
Toggle between i2v or t2v
Postprod
Facedetailer
uses t2v Model + LoRA for inpainting - resources needed included in workflow
Toggle between GIMM VFI and RIFE VFI Interpolation
Upscale
Tensorrt Upscale with Model
Basic Video Upscale with Model
RTX Video Super Resolution Upscale (insanely fast for decent quality)
Frame Clipper
Seamless Loops using custom RIFE nodes https://github.com/Artificial-Sweetener/comfyui-WhiteRabbit
Upscale + Interpolate
I recommend using this workflow instead of upscaling with the generating workflows, since you never really know what kind of results you get, ending up upscaling a bad video and wasting time. I included toggles so you can't use multiple interpolation or upscale nodes at once by mistake.
This includes:
WAN Facedetailer
use any WAN 2.2 T2V low model + the following T2V LoRA:
lower resolution from 768 to 512 if you have VRAM issues
Put the following file into "ComfyUI\models\ultralytics\bbox":
WAN Refiner (massive VRAM cost)
increase denoise if you want more inpainting
Sharpen, Gamma, Brightness and Contrast control
Frame clipper (remove unwanted frames at the start and/or end)
GIMM VFI + RIFE VFI interpolation (I recommend GIMM VFI, much higher quality but also much slower)
Tensorrt Upscale + Basic Video Upscale
both use basic image upscaling models
Tensorrt (faster than Basic Video Upscale) with AnimeSharp4x is recommended for anime
RTX Video Super Resolution Upscale
insanely fast
decent quality
FlashSVR + SeedVR2
experimental
Video upscale models that are more intricate than basic image upscaling models
haven't had great results for anime yet
takes a LOT longer
Saving last frame for manual extensions
mmaudio
added Audio combine node
combine audio from an existing video with the generated audio on top
generate nsfw audio with the nsfw model and then combine that video with another generated audio track from the base model for background noises
removed interpolation for easier and faster audio generation - you have the following options:
upload raw unupscaled video to MMAudio Video node and upscaled video to Combine video node
upload upscaled video to both nodes but lower custom_width and custom_height of the MMAudio video node to about half for faster generation and to prevent VRAM issues
upload raw video to both nodes and upscale afterwards
Inspired by https://civarchive.com/models/2137833
Following resources necessary (ComfyUI\models\mmaudio):
https://huggingface.co/Kijai/MMAudio_safetensors/resolve/main/mmaudio_synchformer_fp16.safetensors
Description
reworked Facedetailer to be more modular
now uses any WAN 2.2 T2V model for much better quality
the modularity allows you to copy the Facedetailer into any other workflow
added Base Step multiplier for 3 samplers
mutiply the base steps to increase quality massively
more info inside the workflow
added RTX Video Super Resolution Upscale for incredibly quick upscaling
FAQ
Comments (15)
Hi, I'm trying to make a video where the character in the initial frame has their back to the camera, then turns around and looks at the camera, but when they do, it's not the same character as in the Anchor/sample image. Could someone please help me? https://ibb.co/60d8mvMq
Anchor samples are not a 100% guarantee since SVI only takes features from that image, different seeds can help but other LoRAs will also influence the outcome. If you want to be 100% sure then you need to use a FLF2V subgraph with an end_sample with the character clearly visible.
Hi, thank you so much for your quick response. I'll try changing to a more visible image and changing the seed.
Hey, could you take a look at my post? I got a good result using your SVI module. Your workflow uses FLF2V, but I couldn’t get it to work as well (might be user error on my side). I also uploaded another workflow and the result from it. Do you think it could be useful for improving your workflow? Image posted by shlgf
You may need to install these nodes manually to get the FLF2V subgraphs to work https://github.com/Well-Made/ComfyUI-Wan-SVI2Pro-FLF
Otherwise you just need to connect the FLF2V subgraphs as you would the normal ones, you can just copy and insert them at any point.
Thanks but my workflow uses a completely different approach to keep it as minimalistic as possible while not giving up on the features I want or need.
@LastAssignment I gave it another try and finally got it working! Also, I noticed that ComfyUI versions after v17 break the external LoRA selection feature in the workflow, so I chose to downgrade. I also ran into one more issue, and I'm not sure if it's just happening on my end: if I directly reuse the example workflow and make modifications on top of it, there's a brief color shift during the transitions between SVIs.
@shlgf Awesome! ComfyUI is constantly breaking everything, I really recommend having atleast 2 different versions, one older portable (I'm running 0.12.3) and whatever else you need to use newer features or whatever.
When exactly do you get the color shifts? Do you use different anchor_samples or end_frames? Do you use the base WAN 2.2 model or some pre baked with lightning LoRAs on top?
I really recommend going as barebones as possible, because any model that has a lot of pre baked LoRAs will have issues every now and then with SVI, especially when going for longer form videos. Also lower the strength of LoRAs as much as possible, especially high LoRAs tend to affect color drift/transformation.
@LastAssignment What I mean is, if I use the "example video" file from the zip as a base, the generated video gets color shifts. The another file works perfectly.
This is great, what an amazing effort! I was losing interest in Wan since the 6 second limit was becoming too restricting so this has been a godsend!
One strange issue I've been encountering is, after running the workflow once, when I try to run it again I get an error in the "LatentBus" subgraph's "motion_latent_count" node saying that the required input is missing. I've been able to work around this by manually disconnecting & reconnecting the "motion_latent_count" noodle to the "LatentBus" node in the "DataBus" subgraph after each run.
I'm speculating that this means that the variable is getting unset after each run and I'm "re-initializing" it by reconnecting the noodle. Have you ever encountered this behavior before?
First time seeing that issue, what ComfyUI version are you running? I'd guess the nested subgraphs could be an issue depending on the ComfyUI version, you can try unpacking the latent subgraph, maybe that helps?
@LastAssignment I'm using the last Comfy as of 24-ish hours ago.
Yeah it's very strange, I have to imagine that something about comfy's canvas is not playing well with subgraphs. I'll try unpacking it and report back. Your workflow is incredible, but with this issue I can't queue up jobs since i have to manually fix the noodle every time.
@TwoMoreLurker I did try the latest 0.18.2 when troubleshooting a new version of my workflow and that ComfyUI version broke everything completely, lmao. I'll definitely stick to my trusty 0.12.3 which I've thrown everything at so far and it works just fine. If you want to be 100% sure then I recommend getting a 0.12.3 portable, that's atleast what I'm running atm.
ComfyUI is known for breaking things every couple of updates :(
@LastAssignment Yeah you're telling me. I think this is about the 3rd time I've borked my Comfy by upgrading, you'd think I'd learn by now.
The good news is that unpacking the "LatentBus" subgraph seems to have alleviated the issue and jobs are now queuing as expected. Thanks again for the amazing workflow!!
"WAN Facedetailer
use any WAN 2.2 T2V low model + the following T2V LoRA: ..."
Is the use of the WAN 2.2 T2V low model (FP16 - 26 GB model or whatever else) actually mandatory?
Because I'm only using the WAN 2.2 T2V LoRA and the faces seem to be corrected with this alone too.
Did you try this?
Using WAN 2.2 I2V - FP16 weights atm.
It isn't mandatory but the quality of the inpaint will be better when using a T2V model. You can just insert a I2V model if the results are good enough for you, it will work just fine. You can play around with the denoise and lightning LoRA strength values to change how the inpainting works.