Initially built with photoshootings and music video clips in mind, the workflow actually automates the production of any type of long cinematic sequence. If you have a story to tell with your pictures, this is the workflow you need.
Main Features
Fully automated from start to end.
Handles from 2 to a potentially unlimited number source pictures.
Flexible prompting.
Flexible video lengths (v2).
Increased generation speed (v2).
Can produce looping and non-looping videos.
Works on 12GB VRAM cards and higher.
Getting Started
Replace the models, clip, and LoRas by those you usually use.
Set the folder location of your pictures, in the node Load Images from Folder (KJ).
Set the outputs width and length(s).
Write your prompt(s)
Click "Run".
Usage & Tips
By default, the same prompt is used for all generations, but you can use different prompts instead, by connecting the Concatenate node to the Positive Prompt node.
If Wan makes unwanted hard cuts, it probably means there is too much "distance" between the start and the end frame. You can increase the number of frames, increase the number of steps, and/or increase the lightx2v LoRas strength to avoid that effect, to a certain extent. On the contrary, if the character barely moves, decrease the number of frames to make him move faster.
(v1) For a non-looping video, disconnect the count connector in the Load Images From Folder (KJ) node from the total connector in the For Loop Start node, and set the total as “number of images - 1”.
Images are sorted by name, so if you need to rename your images, remember that “10” comes before “9” (and before “2”), but “09” comes before “10”.
An example prompt is provided, but feel free to experiment alternative prompts and settings. Enjoy, and if you are satisfied with the result, post your best takes.
Custom Nodes
Required
Optional
ComfyUI-Impact-Pack (for multi-prompting) (v1)
ComfyUI-GIMM-VFI (for interpolation)
Models Used
Wan 2.2 14B I2V, Quantized:
• https://huggingface.co/QuantStack/Wan2.2-I2V-A14B-GGUF/tree/main
Lightx2v LoRas :
• https://civarchive.com/models/1585622?modelVersionId=2361379
• https://civarchive.com/models/1585622?modelVersionId=2014449
Fun LoRa:
• https://huggingface.co/alibaba-pai/Wan2.2-Fun-Reward-LoRAs/tree/main
Description
Improved Speed: by sampling at a low resolution with the high-noise model before upscaling the latent.
Added Flexible video lengths.
Visually cleaner than v1.
Refined default settings.
Dependencies: added RES4LYF custom nodes, removed ImpactPack.
FAQ
Comments (18)
Which value can I adjust to make the loop/video longer? Or do I just need to add more pictures to the flow?
It is the node "Lengths", just next to the Ksampler for high-noise. By default it is set to 33 frames.
You can have only one value or mulitple values (1 per line), both solutions will work.
Nice, is there something else needed to be changed when making it longer because if I put more than 33 then I get this grey flicker at the end of the loop? @qdr1en
@thongraegu I could reproduce your issue when I inserted random figures in the Lengths node ("55", "38").
The values you put in that node must follow the pattern "4x+1" (like 53, 33, 49, 81, 57, 65, etc.). They can't differ from the values allowed in the WanFirstLastFrameToVideo node.
This is an incredibly cool workflow! Thanks a lot to the author!
I have an RTX 5090, and after several successful generations in a big series, your scheme regularly crashes due to (/ComfyUI/run_comfy_clean.sh: line 29: 1794934 Killed /home/goga/miniconda/envs/comfyui/bin/python main.py). I used full WAN 14B models and also try GGUF Q8/Q5, but ComfyUI still crashes unexpectedly without any described error.
Most likely, it's a VRAM overflow. I'd like to know if it's possible to add a memory cleanup node somewhere in the scheme; could that help? Does anyone else have a similar issue?
I had a similar problem after upgrading ComfyUI (have you upgraded recently?).
I could solve the issue by adding "--disable-smart-memory" argument in the .bat file.
@qdr1en No, that doesn't help me. The workflow eats up RAM, but my VRAM is fine. The workflow lacks functionality to clean up RAM between iterations. Because RAM constantly accumulates, and by batch 6 or 7, the process is killed due to running out of free RAM memory. Any ideas?
@anitadpino79994 OK well, if you are on Windows, have you set up a pagefile (ie, https://mcci.com/support/guides/how-to-change-the-windows-pagefile-size/)? I am not sure how to cleanup RAM in ComfyUI, but with 32GB RAM and a big pagefile I never ran into RAM overload issues.
Broken workflow with 3 Images!
Requested to load WanVAE
loaded completely; 4528.47 MB usable, 242.03 MB loaded, full load: True
!!! Exception during processing !!! invalid literal for int() with base 10: ''
Traceback (most recent call last):
File "/home/eddie/comfy/ComfyUI/execution.py", line 510, in execute
output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, hidden_inputs=hidden_inputs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/eddie/comfy/ComfyUI/execution.py", line 324, in get_output_data
return_values = await asyncmap_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, hidden_inputs=hidden_inputs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/eddie/comfy/ComfyUI/execution.py", line 298, in asyncmap_node_over_list
await process_inputs(input_dict, i)
File "/home/eddie/comfy/ComfyUI/execution.py", line 286, in process_inputs
result = f(**inputs)
File "/home/eddie/comfy/ComfyUI/custom_nodes/comfyui-easy-use/py/nodes/logic.py", line 1360, in convert
params = int(anything)
ValueError: invalid literal for int() with base 10: ''
Prompt executed in 69.10 seconds
I ran the workflow yesterday and it worked fine. Try upgrading the custom nodes to the latest version, as the recent ComfyUI releases broke many things.
Have you ever gotten this error? This happens if i try to do more then a single image in the batch...
!! Exception during processing !!! CUDA error: invalid argument
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.
Traceback (most recent call last):
File "D:\comfyui\resources\ComfyUI\execution.py", line 510, in execute
output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, hidden_inputs=hidden_inputs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\comfyui\resources\ComfyUI\execution.py", line 324, in get_output_data
return_values = await asyncmap_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, hidden_inputs=hidden_inputs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\steph\Documents\ComfyUI\custom_nodes\comfyui-lora-manager\py\metadata_collector\metadata_hook.py", line 165, in async_map_node_over_list_with_metadata
results = await original_map_node_over_list(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\comfyui\resources\ComfyUI\execution.py", line 298, in asyncmap_node_over_list
await process_inputs(input_dict, i)
File "D:\comfyui\resources\ComfyUI\execution.py", line 286, in process_inputs
result = f(**inputs)
^^^^^^^^^^^
File "D:\comfyui\resources\ComfyUI\nodes.py", line 295, in decode
images = vae.decode(samples["samples"])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\comfyui\resources\ComfyUI\comfy\sd.py", line 752, in decode
out = self.process_output(self.first_stage_model.decode(samples, **vae_options).to(self.output_device).float())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
torch.AcceleratorError: CUDA error: invalid argument
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.
Hi. I re-ran the workflow today but it still works on my end (ComfyUI v0.4.0+custom nodes upgraded to their latest version).
Recent ComfyUI updates broke many custom nodes though. I can see in your log that the error comes from "comfyui-lora-manager", which is not included in the workflow. Have you tried upgrading it or disabling it?
I can't seem to get RES4LYF to install on the latest ComfyUI manager. Any idea why?
Difficult to know. Check the logs at ComfyUi start.
But you can use the v1 of the workflow instead ; it does not need RES4LYF.
How can I use FP8 models with this? I tried replacing the GGUFs with a load diffusion model but the resulting videos has super high noise level.
Hi. Maybe try to increase the number of steps on the low-noise model, or use the v1 to see if it makes a difference?
Why Lenght list is set as 33 37 33 33 ? If it's 16fps, shouldn't it be a multiple of 16?
The length must follow the pattern "4n+1". The default is 81 (5 seconds).
Disconnect the "length" input from the node WanFirstLastFrameToVideo to see all the available values.