Wan 2.2 largely works if you use the wan2.2_t2v_low_noise_14B file for the Model Loader node and has a much more photorealistic look. It also seems to significantly reduce the color drift if you keep resolution above 720p. Wan 2.1 seems better for loras and a more neutral look though. Specifically Wan 2.1 1.3B seems best for loras if you are trying to do something drastically different, as it seems the higher models are more rigid and locked down.<hr />This is a workflow I posted earlier on Reddit/Github: <a target="_blank" rel="ugc" href="https://www.reddit.com/r/StableDiffusion/comments/1k83h9e/seamlessly_extending_and_joining_existing_videos/">https://www.reddit.com/r/StableDiffusion/comments/1k83h9e/seamlessly_extending_and_joining_existing_videos/</a><a target="_blank" rel="ugc" href="https://github.com/ali-vilab/VACE/issues/45">https://github.com/ali-vilab/VACE/issues/45</a>It exposes a somewhat understated feature of WAN VACE, which is the temporal extension. It is underwhelmingly described as "first clip extension." Even worse, the first clip example VACE devs posted only does image-to-video because their example input is just a single frame. In actuality, it can auto-fill pretty much any missing footage in a video that is masked out - whether it's full frames missing between existing clips or things masked out (faces, objects). It seems the VACE devs themselves do not fully get what their model can do, since none of their examples explore this.It's better than Image-to-Video / Start-End Frame because it maintains the motion from the existing footage (and also connects it to the motion in later clips).Watch this video to see how the source video (left) and mask video (right) look. The missing footage (gray) is in multiple places, missing face, etc that is all then filled out by VACE in one shot.<div data-youtube-video><iframe width="640" height="480" allowfullscreen="true" autoplay="false" disablekbcontrols="false" enableiframeapi="false" endtime="0" ivloadpolicy="0" loop="false" modestbranding="false" origin playlist src="https://www.youtube.com/embed/47i3kdwnpXo" start="0"></iframe></div>It takes in two videos, your source video with missing frames/content in gray and a mask video that is black-and-white (the missing gray content recolored to white). I usually make the mask video by setting brightness to -999 or something to that effect on the original while recoloring the gray to white.Make sure to keep it at about 5-seconds to match Wan's default output length (81 frames at 16 fps or equivalent if the FPS is different). You can download VACE's example clip here for the exact length and gray color (#7F7F7F) to use on the source video: <a target="_blank" rel="ugc" href="https://huggingface.co/datasets/ali-vilab/VACE-Benchmark/blob/main/assets/examples/firstframe/src_video.mp4">https://huggingface.co/datasets/ali-vilab/VACE-Benchmark/blob/main/assets/examples/firstframe/src_video.mp4</a>In the workflow itself, I recommend setting Shift to 1 and CFG around 2-3 so that it primarily focuses on smoothly connecting the existing footage. I found that having higher numbers introduced artifacts sometimes.Tips to maximize video quality and minimize loss of details or color-drifting:<ul><li>Keep CFG 2-3 and Shift=1 to retain as much detail from the existing footage as possible.</li><li>Render at 1080p resolution to minimize color drift. CausVid helps reduce the render time by over 5x (8 steps instead of 50).</li><li>Color Match node in ComfyUI on MKL setting to get it reduced (not always applicable if the scene changes a lot).</li><li>Post correct in video editor the hue by about 2-7 and desaturate a little bit to counteract the drift.</li><li>Starting the scene initially with regular I2V when possible (no color drift) and masking in new changes with VACE (with feathering to blend pieces in and use as much as the I2V scene as possible with no color drift). Alternately extending in FramePack with Video Input or SkyReels V2 as well to get a "skeleton" of the scene without color drift and then patching changes in with VACE.</li></ul>Models to download:<ul><li>models/diffusion_models: Wan 2.1/2.2 T2V (Pick 1, VACE's 14B/1.3B below): Wan 2.2 T2V Low Noise 14B FP16: <a target="_blank" rel="ugc" href="https://huggingface.co/Comfy-Org/Wan_2.2_ComfyUI_Repackaged/blob/main/split_files/diffusion_models/wan2.2_t2v_low_noise_14B_fp16.safetensors">https://huggingface.co/Comfy-Org/Wan_2.2_ComfyUI_Repackaged/blob/main/split_files/diffusion_models/wan2.2_t2v_low_noise_14B_fp16.safetensors</a> Wan 2.2 T2V Low Noise 14B FP8: <a target="_blank" rel="ugc" href="https://huggingface.co/Comfy-Org/Wan_2.2_ComfyUI_Repackaged/blob/main/split_files/diffusion_models/wan2.2_t2v_low_noise_14B_fp8_scaled.safetensors">https://huggingface.co/Comfy-Org/Wan_2.2_ComfyUI_Repackaged/blob/main/split_files/diffusion_models/wan2.2_t2v_low_noise_14B_fp8_scaled.safetensors</a> Wan 2.1 14B FP16: <a target="_blank" rel="ugc" href="https://huggingface.co/Comfy-Org/Wan_2.1_ComfyUI_repackaged/blob/main/split_files/diffusion_models/wan2.1_t2v_14B_fp16.safetensors">https://huggingface.co/Comfy-Org/Wan_2.1_ComfyUI_repackaged/blob/main/split_files/diffusion_models/wan2.1_t2v_14B_fp16.safetensors</a> Wan 2.1 14B FP8: <a target="_blank" rel="ugc" href="https://huggingface.co/Kijai/WanVideo_comfy/blob/main/Wan2_1-T2V-14B_fp8_e4m3fn.safetensors">https://huggingface.co/Kijai/WanVideo_comfy/blob/main/Wan2_1-T2V-14B_fp8_e4m3fn.safetensors</a> Wan 2.1 1.3B FP16: <a target="_blank" rel="ugc" href="https://huggingface.co/IntervitensInc/Wan2.1-T2V-1.3B-FP16/blob/main/diffusion_pytorch_model.safetensors">https://huggingface.co/IntervitensInc/Wan2.1-T2V-1.3B-FP16/blob/main/diffusion_pytorch_model.safetensors</a> Wan 2.1 1.3B FP8: <a target="_blank" rel="ugc" href="https://huggingface.co/Kijai/WanVideo_comfy/blob/main/Wan2_1-T2V-1_3B_fp8_e4m3fn.safetensors">https://huggingface.co/Kijai/WanVideo_comfy/blob/main/Wan2_1-T2V-1_3B_fp8_e4m3fn.safetensors</a></li><li>models/diffusion_models: WAN VACE (Pick 1, Match Wan's 14B/1.3B above) 14B BF16: <a target="_blank" rel="ugc" href="https://huggingface.co/Kijai/WanVideo_comfy/blob/main/Wan2_1-VACE_module_14B_bf16.safetensors">https://huggingface.co/Kijai/WanVideo_comfy/blob/main/Wan2_1-VACE_module_14B_bf16.safetensors</a> 14B FP8: <a target="_blank" rel="ugc" href="https://huggingface.co/Kijai/WanVideo_comfy/blob/main/Wan2_1-VACE_module_14B_fp8_e4m3fn.safetensors">https://huggingface.co/Kijai/WanVideo_comfy/blob/main/Wan2_1-VACE_module_14B_fp8_e4m3fn.safetensors</a> 1.3B BF16: <a target="_blank" rel="ugc" href="https://huggingface.co/Kijai/WanVideo_comfy/blob/main/Wan2_1-VACE_module_1_3B_bf16.safetensors">https://huggingface.co/Kijai/WanVideo_comfy/blob/main/Wan2_1-VACE_module_1_3B_bf16.safetensors</a></li><li>models/text_encoders: umt5-xxl-enc (Pick 1): BF16: <a target="_blank" rel="ugc" href="https://huggingface.co/Kijai/WanVideo_comfy/blob/main/umt5-xxl-enc-bf16.safetensors">https://huggingface.co/Kijai/WanVideo_comfy/blob/main/umt5-xxl-enc-bf16.safetensors</a> FP8: <a target="_blank" rel="ugc" href="https://huggingface.co/Kijai/WanVideo_comfy/blob/main/umt5-xxl-enc-fp8_e4m3fn.safetensors">https://huggingface.co/Kijai/WanVideo_comfy/blob/main/umt5-xxl-enc-fp8_e4m3fn.safetensors</a></li><li>models/vae: WAN 2.1 VAE (all WAN versions): <a target="_blank" rel="ugc" href="https://huggingface.co/Comfy-Org/Wan_2.1_ComfyUI_repackaged/blob/main/split_files/vae/wan_2.1_vae.safetensors">https://huggingface.co/Comfy-Org/Wan_2.1_ComfyUI_repackaged/blob/main/split_files/vae/wan_2.1_vae.safetensors</a></li><li>models/loras: WAN CausVid V2 14B T2V, reduces steps to 8 (for Wan 2.1 14B only): <a target="_blank" rel="ugc" href="https://huggingface.co/Kijai/WanVideo_comfy/blob/main/Wan21_CausVid_14B_T2V_lora_rank32_v2.safetensors">https://huggingface.co/Kijai/WanVideo_comfy/blob/main/Wan21_CausVid_14B_T2V_lora_rank32_v2.safetensors</a></li></ul>Tutorial/walkthrough videos:<div data-youtube-video><iframe width="640" height="480" allowfullscreen="true" autoplay="false" disablekbcontrols="false" enableiframeapi="false" endtime="0" ivloadpolicy="0" loop="false" modestbranding="false" origin playlist src="https://www.youtube.com/embed/0gx6bbVnM3M" start="0"></iframe></div><div data-youtube-video><iframe width="640" height="480" allowfullscreen="true" autoplay="false" disablekbcontrols="false" enableiframeapi="false" endtime="0" ivloadpolicy="0" loop="false" modestbranding="false" origin playlist src="https://www.youtube.com/embed/_fmc-Ovh5CU" start="0"></iframe></div>

Wan VACE Masking & Extension - Seamlessly Extend, Join, and Auto-Fill Existing Videos While Maintaining Motion

seamlesslyExtendingAndJoining_v10.zip

Files with SHA256: 79b94407d076d44b3c07c8a28a2fbda5dc0b8678c95215919c714ef26e045a36