⚠️ The I2V Vid extend workflow is highly experimental and requires high end GPUs.
I've created this Wan Video Image2Video workflows based on Kijai's workflow and the Native ComfyUI workflow, added an easy to use interface along with upscaling and frame interpolation.
Key Features:
1st pass - generating vanilla video
2nd pass upscaling using user selected upscaler and frame interpolation to 32FPS
This workflow comes pre loaded with my Wan14B RunPod template:
https://civarchive.com/articles/11960/wan14b-in-1-click-with-workflows-included-runpod-template
Join my Discord server for updates on new LoRAs, tips & tricks, Workflows, RunPod templates and the holy right to be close to an amazing charismatic person like myself:
https://discord.gg/fyha5Pzm
Description
I'm releasing another HIGHLY EXPERIMENTAL version of my I2V workflow.
The main change is another sampling steps that takes the last frame of the first video and extend the video.
I'm able to get somewhat OK results with it.
FAQ
Comments (105)
The Experimental workflow how much VRAM need?
The same as other workflows.
RAM and CPU is tricky here as the frame interpolation node takes much more frames than my other workflows.
I don’t have exact numbers unfortunately
Standard settings gave an OOM on my 3090 (24GB).
fp8 gave same result.
But I have no idea what the hell I'm doing so your milage may vary ;)
Let me correct that, I managed to run this successfully with quantization.
I get blank output when selecting 720p FP8 on load diffusion model and Teacache.
"Where can I download wan female masturbation?"
Can you actually add the node to download the last frame ? :)
This workflow is great. I replaced model loader with the UnetLoaderGGUFDisTorchMultiGPU and offloaded like 4 gb ram. Now the generations are way faster for a 4080.
Sounds great, thank you for sharing!
Would you be able to post the modified workflow? I'm trying to figure out what the gguf loader can be connected to, but am stumped.
i would love to see the workflow aswell
@Mitch_Connor_420_69 @Atti69 I don't think it is worth posting the workflow, I just modified two nodes. I will just DM you guys.
@CheeseCoffee please just post here there are more than 2-3 ppl with a 4080 gpu
@MetaGen Here you go
https://www.mediafire.com/file/nb278qkf9l102f8/titdrop_GGUF_Q4_n.json/file
@CheeseCoffee Thank you for this. I only have 16GB of VRAM and now I can use this.
Is there any reason why you don't use Enhance-A-Video?
Didn't test it enough so didn't include it.
I'm planning to do it this week.
@Hearmeman Cool. I think it's worth using based on my tests just FYI.
i2v doesnt seem to work, its ignoring the image completely
The extended version works great! Getting teacache and sageattn2 working is totally worth the effort to boost the speed. It's already decent on a 4090, but I'd love to see how much quicker a 5090 is.
5090 can generate a 480x720 101-frame video in 2 minutes and 30 seconds
Extremely slow, 340s/it on 4080 without tweaking anything, do not bother without a cluster.
I'm using your one click runpod currently, but is there a way to download the GGUF version of the models to use? I tried uploading it but it takes way too long.
Use wget.
Hey, nice work on this it's very clean. Main question: where do I put film_net_fp32.pt to get that running? Minor question: have you tried adding SkipLayerGuidance? I tried DIYing it and the lady get fucked set on fire randomly lol.
I was just about to try Skiplayerguidance too. now im scared lmao
@dkain76 see if you have better luck than me!
Put it in the upscale modes folder
Thanks that works. Weirdly FILM OOMs me on a 3090. Seems the VRAM isn't getting cleared before the FILM node, even if I add a GPUCleanup node before it. Guess I'll just figure out doing the interpolation as a separate wf.
Unet Loader (GGUF) with Q4_K_M quant replacing "Load Diffusion Model" allows the default config to run great (12 mins) on 12GB VRAM with sage and teapot.
Would it be possible to use a second prompt for the extended clip? I.e. the initial prompt as it is now, then it takes that frame to start the second clip, but using a different prompt to change the content somewhat of the second half of the clip?
As it is now, it's more or less going to generate one extended clip that follows that prompt, which is fine for static sex scenes or whatnot, but you could probably get more control with a second prompt. I.e first prompt missionary, second prompt pulling out, etc.
You can, you just have to modify the workflow.
Don't understand what i am doing wrong. Other workflows seem to follow my prompts way better. Here, when i make 10 clips, 9 of them are almost only still frames, or tiny amounts of motion. 1 does what i want and has a lot of motion.
It's so weird how it's either "almost no motion" or "a lot of motion". There seems to be no middle ground. As if the generator rolls a dice and then makes a video or a still image depending on the outcome.
Please share your settings
Is it possible to use swap blocks with the extended version? Without swap blocks initial start of generation seems to take forever on my 3090
Has anyone else recently been getting the -transformer_options not found in extra_args, currently SkipLayerGuidanceWanVideo only works with TeaCacheKJ - error? I was using it with no issues a day ago.
Yes, the newest comfyui update/node updates seemed to have messed with SkipLayerGuidanceWanVideo. You can disable SkipLayerGuidanceWanVideo and the workflow will work but the generations are less quality. Gonna just wait for an update on the workflow.
@endnote Thanks mate!
@endnote any update yet? would like to know if i need find other workflow...... thanks!
@Tartrixroar You don't need to find another workflow.
This is a bug with these nodes and not with the workflow.
Anyway, Kijai released an update that should fixed this, update your KJNodes custom nodes.
@Hearmeman [Installation Errors] 'ComfyUI-WanVideoWrapper': With the current security level configuration, only custom nodes from the "default channel" can be installed
@Tartrixroar Install the WanVideoWrapper for the git repo, not from the manager.
@NeuralDesires @endnote
This is fixed, update your KJNodes
@Hearmeman I updated the kjnodes from github - cloning and installing the dependencies, still getting the same error with SkipLayerGuidanceWanVideo only works with TeaCacheKJ. I also decided to re-install the wanvideowrapper but nothing changed
And what system requirements is workflow designed for? I have it in my bag with RAM and SWAP, it doesn't start on 96 GB RAM.
And if it does start, the generation takes about an hour.
4070 ti super
I'm struggling with "easy clean gpu used". I install it through the manager but when I reload it says I don't have it. suggestions? I'm a comfyui noob.
You might need to install it directly from the repo.
You can also just delete it
@Hearmeman Thanks, figured it out. Just needed to run the update thingy for comfyui.
Getting the following error "transformer_options not found in extra_args, currently SkipLayerGuidanceWanVideo only works with TeaCacheKJ". All nodes are updated
Run git pull on KJ nodes
@Hearmeman Its on latest commit, but still seeing error
@expert6699 Hmm, not sure.
You could try the set the start step of SLG to 0.3 and end step to 0.7, this is the config that works for me.
I got the same error, temporarily switched to "WanVideo Tea Cache (native)" (BetaKJnodes) and then it started working again, will try an update.
in the WanVideoLoader if you dont enable Quantization (i used fp8_e4m3fn) it use 46GB VRAM...
turning this on lowered it to 17.1GB Vram
Thanks for the workflow! Could you please explain why my videos always have different first frame? I would like the first frame to always be the same as the original image. Is this possible?
there's a note that says the first 6 frames are being dropped but i can't understand where or why this is happening. bypassing the 2 nodes broke the workflow. i also want to retain those frames.
I think i solved it, find the Image From Batch node near the top, right click -> Fix node (recreate). Then the inputs will appear (they were invisible for me for some reason) and then you can set the start value to 0 to keep all the frames.
Tried everything but not working after update keep getting this error "transformer_options not found in extra_args, currently SkipLayerGuidanceWanVideo only works with TeaCacheKJ"
You workflow was very good but now I am just hopless
You can just bypass the SLG node, you don't have to be hopeless :)
Anyways, update KJ nodes to the latest version with git pull.
Set the start_at at 0.3 and end_at at 0.7.
Should work.
@Hearmeman It doesnt work
Try to use WanVideo Tea Cache (native) node instead regular Tea Cache node
@Hearmeman Yes . That works
In the i2v Extend Video workflow, if I set batch_size to 2 in the "Set Video Length/Width/Height WanImageToVideo" node (whose output connects to KSampler (First Pass), it seems to generate two videos as one file, then only extends the second video. Is there any way around this?
Actually, I'm not even sure batch size >1 would actually save time
Runs great!
There are a lot of workflows, wich one is the most updated one with the best quality, the native comfyui ones or the WanVideo ones? the wanvideo ones always give me out of memory errors even on a 4090 when i load the 720p 14b model
They organized by date, the latest one with TeaCache/SageAttn/SLG is the latest and best one IMO.
I also added the WanVideo ones as some people prefer to use them as they are a bit more flexible.
Regardless, I wouldn't run the 720P model on a 4090, too heavy.
@Hearmeman yep, rtx a6000 or a l40 is what I use for the 720. It needs that extra ram, can't run (consistently) without it. I can do 5-15 second single shot videos on the rtx a6000 and the l40 as well. The l40 has more "room" but I haven't tried to do more than 15 seconds at a time. Depending on what you're doing and what your goal is, andhow much you want Wan to do whatever it wants, anything over 10 seconds ends up meandering too far off subject. Also though, I think people are obsessing over video length/extension/initial video output quality when that shouldn't even be the focus. Most motion media is shot using 3-15 second blocks before camera/view change. most things actually run with much quicker changes, especially action movies and music videos. So it is absolutely possible to render an entire 5 minute music video or a 2 hour movie using mostly 5 second or less clips. So the focus, especially with video generation, should be consistency, which will port over to upscale-ability, and consistency comes from proper prompt usage, how to get consistent character/scene images from multiple poses and angles, and how to guide the video generation with either a controlnet type interface or really good prompting + that initial batch of images. IF you are good at prompting flux or IL or whatever and you know your way around face swap and can get a fast enough batch render to get consistent body types, etc... (also has a lot to do with prompting, seed management, etc) you can literally spit out as much video as you want and it be consistent enough to use for most purposes, without needing to extend TOO many scenes. Some shots run long, like landscape establishment scenes, or certain types of dialogue scenes, so there is definitely a need for extension or rendering a longer video from time to time, but i think people are focusing on that more than they should at this point, especially when if they'll sit down with a stop watch and pick any movie or tv show or music video and hit the button every time the scene switches angles or otherwise changes, they'd be surprise just how short a contiguous scene is in typical situations. But as far as output quality of the two, I think the 480 is quite adequate. If you set it to 512 for your narrow panel instead of 480 like many are doing, you actually get pretty good frames out of the box, and it renders pretty quick. 81 frames in something like 7 minutes on the rtx a6000 (give or take, depending on settings). Of course the 720, set at 768 on the narrow panel will render cleaner frames out of the box, which makes upscaling to 1080 much easier but still. Either one would render fantastic results. If you got access to random cards, do the 720. Not so much? do the 480 and upscale. As far as extensions... idk... I find it works best to just render a 6 or 7 second video, and that way I can trim it down to 5 or whatever I need and cherrypick the best part of it.... but, it does take a decent amount of memory even on the 480p to render past 7 or so seconds. Anyway, I'm rambling, but with a purpose, lol so I'm out.
@Hearmeman So i shouls use the native comfyui ones right and not the wanvideo ones, also if i want a less blurry output what should i do? disable teacache or increase steps?
@msiaigens yes
Getting this error from ImageFromBatch+ node
'list' object has no attribute 'shape'. Please help
Traceback (most recent call last):
File "F:\ComfyUI_windows_portable\ComfyUI\execution.py", line 327, in execute
output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\ComfyUI_windows_portable\ComfyUI\execution.py", line 202, in get_output_data
return_values = mapnode_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\ComfyUI_windows_portable\ComfyUI\execution.py", line 174, in mapnode_over_list
process_inputs(input_dict, i)
File "F:\ComfyUI_windows_portable\ComfyUI\execution.py", line 163, in process_inputs
results.append(getattr(obj, func)(**inputs))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_essentials\image.py", line 166, in execute
length = image.shape[0]
^^^^^^^^^^^
AttributeError: 'list' object has no attribute 'shape'
is it required to use sageattention with the sageattention workflow?
No, there is a fast bypasser for sage attention, though I highly recommend finding a way to make it work as it really speeds up generations, but it will work without it, just turn it off via the purple Fast Bypasser node located to the right of the SageAttention node.
I'm using i2v mine looks smudge and lots of artifacts. I tried lowering the tea cache and bypassing slg and there's still slight of the smudge painting artifacts ruining the face of the person. I'm using the default workflow as is and the only thing I change is sageattention to sdpa because even if I tried to installed triton and sageattention another error occurs which kinda making this workflow not working at all with sageattention but it works fine with sdpa. I've used kijais newest version. While lora animates better on this workflow it has that artifact. Kijai however keeps the image clear but the lora isn't following everything so your workflow is ideal for lora. Only that it was creating that artifact and I don't know how to explain. Shall I link an example? btw I tried interchanging unipc and euler, nothing happens. A slight change but still super smudgy.
Incredible Runpod ready workflow. Thank you so much!
for some reason the Runpod version does not work for me at the moment. It can't find the model although in the log it seems to download. Tried twice, gave up, maybe a temporary problem?
did you change the 'false' tag to 'true' for the models you want to download in the Envrionment Variables tab?
@davedrewhull898
Thank you.
I was under the impression that the workflow was ready to run onze Runpod started the pod.
Where do I find the 'Envrionment Variables tab'?
Is this something in ComfyUI or at Runpod?
@tiskloteweer588 It is ready to run.
Before deploying a pod, you have to configure the environment variables to choose which models to download.
I cover this here at 1:00
https://www.youtube.com/watch?v=HAQkxI8q3X0&t=8s&ab_channel=HearmemanAI
@Hearmeman Great thank you!
Using an RTX4090 on Runpod and on the first generation I get Ksampler OOM error and then it works for a while and then I start getting it after a few generations. Any idea what I can do? Which GPU are you using it with on Runpod?
Update: I just started using an RTX 6000 ada and although a bit slower on gen compared to the 4090, I no longer get an OOM errors and can generate a longer clip as well.
Can anybody Help me? The only thing Im missing was Int node ? im still clueless where i can get Node
Comfy Literals
so I am having issues since i updated comfyui where all the connections are not working anymore. I notice that the upscaler isnt connected anymore.
Exactly the same here. The workflows seem broken after playing with LTXV.. Not sure if it is related.
CLIPTextEncode.encode() missing 1 required positional argument: 'clip'
And some connections are missing that I also can not manually connect again.
The workflows where working before the LTXV experiments.
The latest update of Comfyui broke the "anything" and "everywhere" nodes (so possibly part of this workflow, i can't recall, or broke something else), i've had to revert back to previous version myself as it broke other workflows.. See here for details: https://github.com/chrisgoringe/cg-use-everywhere/issues/281#issuecomment-2816364564
LoadWanVideoClipTextEncoder 'NoneType' object has no attribute 'lower'. not sure how to fix this. running locally. would appreciate help. thanks
New Error Now
CLIPTextEncode
CLIPTextEncode.encode() missing 1 required positional argument: 'clip'
anyone notice that the teacache node is not working anymore?? Been trying different parameters but it is not even trigger when I try to fix it
?
Link 381 is invalid, both origin 45 and target 106 do not exist Link 380 is invalid, both origin 16 and target 106 do not exist Link 379 is invalid, both origin 25 and target 106 do not exist Link 378 is invalid, both origin 167 and target 17 do not exist Link 377 is invalid, both origin 167 and target 116 do not exist Link 376 is invalid, both origin 25 and target 110 do not exist Link 375 is invalid, both origin 167 and target 22 do not exist Link 373 is invalid, both origin 167 and target 111 do not exist Link 372 is invalid, both origin 16 and target 117 do not exist 572 is def invalid; BOTH origin node 231 doesn't have 572 and 231 target node doesn't have 572. 571 is def invalid; BOTH origin node 232 doesn't have 571 and 232 target node doesn't have 571. 570 is def invalid; BOTH origin node 232 doesn't have 570 and 232 target node doesn't have 570. 569 is def invalid; BOTH origin node 232 doesn't have 569 and 232 target node doesn't have 569. 568 is def invalid; BOTH origin node 232 doesn't have 568 and 232 target node doesn't have 568. 567 is def invalid; BOTH origin node 232 doesn't have 567 and 232 target node doesn't have 567. 566 is def invalid; BOTH origin node 232 doesn't have 566 and 232 target node doesn't have 566. 565 is def invalid; BOTH origin node 244 doesn't have 565 and 244 target node doesn't have 565. 564 is def invalid; BOTH origin node 243 doesn't have 564 and 243 target node doesn't have 564. 563 is def invalid; BOTH origin node 154 doesn't have 563 and 154 target node doesn't have 563. 562 is def invalid; BOTH origin node 232 doesn't have 562 and 232 target node doesn't have 562. 561 is def invalid; BOTH origin node 233 doesn't have 561 and 233 target node doesn't have 561. 560 is def invalid; BOTH origin node 231 doesn't have 560 and 231 target node doesn't have 560. 551 is def invalid; BOTH origin node 231 doesn't have 551 and 231 target node doesn't have 551. 550 is def invalid; BOTH origin node 232 doesn't have 550 and 232 target node doesn't have 550. 549 is def invalid; BOTH origin node 244 doesn't have 549 and 244 target node doesn't have 549.
Update your CG-use-everywhere node.
@Drabini How are we supposed to know which nodes are required for the workflow? I thought that the workflows could download the nodes necessary?
Is it normal for the workflow to sit at 42% WanVideo Sampler for over an hour>?
No.
Check your settings/hardware
@Hearmeman well i got it kinda working. it renders an image into a video, but it ignores the prompt for what to do with the image. is there a more newb friendly workflow that can accept prompts more readily>?
results with this are insanely good!
Since an update issue with stopping on width and height >
First it was on width I renewed that one and then the error was on height, couldn't get it to work so:
full new install of comfyui + work flow etc.
Same issue.
Simple Math Simple Math.execute() got an unexpected keyword argument 'c'
and I can't get it to work at all anymore.
Same error here. Not sure how to fix.
I keep getting a "current_percent" issue any fixes?
it shows alert that missing node int while loading workflow
same
You need to install the comfyui-logic module via the ComfyUI Manager.
In order to be able to so, you will need to {install-location}\ComfyUI\user\default\ComfyUI-Manager\config.ini and change 'security_level' to 'normal-' e.g. security_level = normal-
So do this with discretion, and consider changing it back afterwards.
If anyone still experiencing this - the nightly build did not work for me. You need to install 1.0.0 via ComfyUI Manager.