Version 1.1
This version adds an image loader for adding key frames. Key frames can be selected by using the camera movement command "Go to image 2" to end on the second image in the key frame selection box. You can still make loops by going to the "original image" or, if you prefer, "go to image 0".
Lightning can now be turned off and on with a single selection, and the number of steps will automatically be adjusted. SVI is also available, but it often has side effects like severe color shifts.
All videos posted were created with lightning on, but turning if off improves maintaining style and smooth animation. If quality is important, test animations with lightning on, but turn it off for the final generation.
By default, interpolation is set to 1, which doubles the rendering speed, but even higher quality is available by setting it to 0.
Version 1.0
This workflow is a variation on the amazing work of https://civarchive.com/user/Usako_USA and their Wan2.2 Highspeed "FreeLength(Loop Processing)" I2V Workflow for Your Scenario (v1.1).
Cosmetic changes are removing the LLM calling for suggestions, and combining several nodes into subgraphs to keep the flow clean. I also added some notes about what certain settings are for.
Functional changes are:
Add a camera movement "Return to original image" that makes the last frame of that segment the original image, useful for loops or to reset quality.
Switch from KSampler to SamplerCustom to allow full sigma control, this reduces image degradation as the video lengthens.
Added a rescale multiplier to the model to slow video degradation even more.
Automated some of the manual settings.
Fixed bug when new image size is not a multiple of 8 that caused the video to jiggle between scenes.
The images contain the workflow settings I use to make the video, just drag them to the ComfyUI window to open.
TL;DR version:
Select the gguf files that fit your system.
Add the starting image.
Update prompt #1 with a description of anything that should remain fixed through the entire video, e.g. "A girl with short blond windswept hair is sitting at a table under a beach umbrella."
Update prompt #2 with the individual scenes as "<time>|<action>|<camera movement>, e.g. "4|The girl holds the glass to her mouth and sips slowly from the straw in the orange juice. |The camera remains steady and doesn't move."
Then run to get the video built.
Camera movement can be holding still, panning, rotating, orbiting, or "Return to original image" to create a relatively seamless loop. Add LoRas for more complex camera movement, or better action adherence in the panels at the far left. Remember to add any LoRa activation keywords to the segments that use it.
Description
v1.0 - Bug fixes and ability to create loops
FAQ
Comments (6)
While I appreciate your simplifying the look of the workflow, I should note to anyone creating subgraphs that it's more helpful to include widget parameters (by making them visible on your subgraph) that people will want to change--like the sampler and scheduler, the cfg settings, the randomization or fixed nature of the seed, etc. For example, Wan 2.2 left me a bit underwhelmed in terms of video quality until I watched a vide by Runpod about how you can improve your results by changing your sampler. (https://www.youtube.com/shorts/lSD817rmrKA ) The video makes no recommendations but shows the results of 20 different samplers for Wan2.2 so the viewer can make their own choices. I found that dpmpp_2m gave excellent results, although it does increase the generation time. Using your workflow, I had to right click on the sampler node to choose to edit the subgraph widgets to show the hidden parameters so I didn't have to open each subgraph to do it that way.
Thanks for the tips. I based the choice of samplers on another video, including the choice of sigma values for lightning. For one thing, it said that only Euler and uni_pc even worked with Wan 2.2, so I really thought these settings would be fixed,or at most changed once. I'll watch the video in your link and try getting an update out.
I don't know why, but the characters in the generated video have absolutely no resemblance to the pictures.
Make sure the text in "Positive Prompt #1" is valid for the entire video. If you say a person is sitting in that box, and then have an action of them standing up, it completely messes up the prompting pattern.
Another possibility is that the image isn't a photo. Lightning was trained solely on real images, so while you can use it to test the prompts for action, you have to turn it off and make the video slowly with the base Wan2.2 to get good animation most of the time.
I have had similar issues, which I'd tracked back to myself using a T2V model instead of a T2I one. Not saying that this is your case, but figured it worth mentioning, since I've actually done this more than once (LOL), not to imply you'd make as doofy a mistake as I, but, just in case we're similarly human... :)
@SkyDood Oh, I've made far doofier mistakes than that. That's why I keep trying new ways to make sure there are fewer ways to mess up in this workflow.