CivArchive
    ComfyUI Txt2Video with SVD - v1.3
    Preview undefined

    UPDATE: You can download SVD ver 1.1 here: https://huggingface.co/stabilityai/stable-video-diffusion-img2vid-xt-1-1/tree/main

    "This model was trained to generate 25 frames at resolution 1024x576 given a context frame of the same size, finetuned from SVD Image-to-Video [25 frames]."

    From Stable Video Diffusion's Img2Video, with this ComfyUI workflow you can create an image with the desired prompt, negative prompt and checkpoint(and vae) and then a video will automatically be created with that image.

    v1.3_sd3: txt2video with Stable Diffusion 3 and SVD XT 1.1

    For information where download the Stable Diffusion 3 models and where put the .safetensors: https://civarchive.com/models/497255/stable-diffusion-3-sd3

    v1.3_cascade: txt2video with Stable Cascade and SVD XT 1.1

    For information where download the Stable Cascade models and where put the .safetensors: https://gist.github.com/comfyanonymous/0f09119a342d0dd825bb2d99d19b781c

    v1.3: svd default is now: svd_xt_1_1.safetensors

    .gif creation instead .webp(.mp4 is still present)

    v1.2: changed the default value of fps svd img2vid_conditioning from 8 to 10, previously it was erroneously different from the save value and save webp value of 10

    v1.1: more clear interface

    v1: first version

    Required:

    1- on Windows: 8+ GB VRAM NVIDIA gpu only

    on Linux: 8+ GB VRAM NVIDIA gpu and AMD gpu

    2- ComfyUI: https://github.com/comfyanonymous/ComfyUI

    3- this worfklow

    4- in ComfyUI\models\checkpoints:

    https://huggingface.co/stabilityai/stable-video-diffusion-img2vid/blob/main/svd.safetensors <-- for 14 fps video

    or

    https://huggingface.co/stabilityai/stable-video-diffusion-img2vid-xt/blob/main/svd_xt.safetensors <-- for 25 fps video

    5- a checkpoint in ComfyUI\models\checkpoints

    6- a vae in ComfyUI\models\vae (optional)

    7- ComfyUI Manager for the Missing Nodes: https://github.com/ltdrdata/ComfyUI-Manager

    P.S.: The recommended resolution for SVD is 576x1024, but in the workflow I set a default value of 512x768 (with 40 steps, 20 steps is good enough). It's up to you to choose the resolution and steps that you want... but 512x768 with 40 steps is a good starting point for your attempts.

    If you want to support me, check out my Patreon: https://www.patreon.com/Shiroppo

    Follow me on civitai(!!!): https://civarchive.com/user/shiroppo

    Follow Me on Twitter/X: https://x.com/ShiroppoTwit

    Pixiv: https://www.pixiv.net/users/96490888

    ---

    Description

    - svd default is now: svd_xt_1_1.safetensors

    - .gif creation instead .webp(.mp4 is still present)

    FAQ

    Comments (12)

    thetokerNov 27, 2023
    CivitAI

    Got it working, interesting stuff. Are there more tips or a more comprehensive guide on using this? How do you control the length of the video? Your default settings has 25 frames at 8 fps in the img2vid_conditioning node but 10fps in the save and video combine nodes, any reason why they differ? Is there any way to control what the video does? I generated one of a woman on a street and the video was a static picture of the woman with a background moving around her. Another was just a static pan from left to right. Interesting but a bit strange.

    shiroppo
    Author
    Nov 27, 2023· 3 reactions

    https://stability.ai/news/stable-video-diffusion-open-ai-video-model <-- official svd page

    - 8 fps in the img2vid_conditioning it was my mistake during the initial experiments, corrected in v1.2 with the value of 10, thank you very much

    - the length of the video is frame / fps so the default value is 25/10(in save, save webp and conditioning)=2 seconds and half... if you try 5 fps you will have a 5 second video, but above 4 seconds the quality drops a lot, unfortunately this model svd is only made for very short videos, I hope stability ai will create new models in the future for longer videos

    - for the prompt, you have to be precise about which part of the body to move, an example is "1girl, wind on hair", that's also why a few days ago I had thought about making it easier for example making a txt2img2img2video, with 2 prompts instead of 1 but the results were not good, for now writing a prompt+an action is the only solution

    GnomeExplorerNov 28, 2023· 2 reactions

    As far as the frame length like the link said it's pretty restrictive right now, the models are trained for 14 frames with regular SVD and 25 frames with SVD-XT.

    The (untried) solution to that limit would just be to save a sequence and just use one of the looping nodes and a conditional to feed the last frame of the video the model generated back into another img2video to produce another 25 frames and repeat as desired.

    GnomeExplorerJan 6, 2024

    I tried doing the feedback of the last image into another ksampler thing finally with an img2video... as long as your last frame was stable (sometimes it goes weird on you) it'll usually keep generating smoothly. If there's a ton of motion it might bounce back in a weird way or start doing something else but if the image is similar enough to the first it'll work. So keep motion lower maybe? It would be harder with txt2video since you can't easily inject the prompt conditioning into the conditioning the image produces. I just kept getting mismatched tensor sizes regardless of attempts to forcibly resize the conditionings. It's not predictable but it's fast enough that I don't mind. I feel like this process would work better with animatediff video2video where the image feedback from the first sampler probably wouldn't even be required so it could be set up as a loop more easily and allow for more optimal batch sizes for your GPU to be run with the same results even if you kept the length the same.

    I used qq-nodes-comfyui's "image accumulator start" and "image accumulator end" to gather all the images produced, set count to 49, fed the first 24 of the batch from ksampler 1 into that, the last one into a clip setup for the second ksampler and then fed those 25 into the accumulator node which trips the count and sends it through the end node (needed a cr image list to batch, nothing else worked for some reason) to the animation output.

    MegasherruNov 28, 2023· 1 reaction
    CivitAI

    Very cool, Just tested it myself and it works super well

    matthew70Nov 29, 2023
    CivitAI

    could do make the sampler of generating video being lcm

    th7rabiDec 28, 2023· 1 reaction
    CivitAI

    I keep getting this error. Can you help me?

    ERROR:root:!!! Exception during processing !!!

    ERROR:root:Traceback (most recent call last):

    File "D:\AI\ComfyUI\execution.py", line 153, in recursive_execute

    output_data, output_ui = get_output_data(obj, input_data_all)

    File "D:\AI\ComfyUI\execution.py", line 83, in get_output_data

    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)

    File "D:\AI\ComfyUI\execution.py", line 76, in map_node_over_list

    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))

    File "D:\AI\ComfyUI\comfy_extras\nodes_video_model.py", line 45, in encode

    output = clip_vision.encode_image(init_image)

    AttributeError: 'NoneType' object has no attribute 'encode_image'

    EpockDec 31, 2023· 1 reaction
    CivitAI

    Thanks that's exactly what was lacking for img2vid

    q8549834457Jan 24, 2024· 1 reaction
    CivitAI

    RuntimeError: Conv3D is not supported on MPS
    How to solve this problem? My computer is a new type of Mac computer

    shiroppo
    Author
    Jan 24, 2024
    q8549834457Jan 24, 2024· 1 reaction

    @shiroppo OK thanks👌

    Workflows
    SVD XT

    Details

    Downloads
    3,636
    Platform
    CivitAI
    Platform Status
    Available
    Created
    2/10/2024
    Updated
    7/27/2026
    Deleted
    -

    Files

    comfyuiTxt2videoWith_v13.zip

    Mirrors

    CivitAI (1 mirrors)