Notice: Due to the lack of updates for AnimateDiff CLI I can no longer offer support for DiffEx or guarantee its functionality.
DiffEx is an easy to use Desktop UI for AnimateDiff CLI Prompt Travel. It takes out a lot of the headache and labor needed to create animations by doing most of the work for you. A detailed readme is included to get you up and running!
New Features in v1.5
Console-less! A cool new progress bar has taken the place of the console for a less cluttered user experience.
Folder walking! Any sub folders in your Stable Diffusion directories will now be detected and added to DiffEx.
Added a refresh button so you no longer have to restart DiffEx after downloading new models.
Added the ability to use a video as an IP-Adapter for better coherence especially when using Stylize.
Added the Canvas to the Region tab so you can now draw your masks inside of DiffEx.
Added a Separate feature to Regions for better layer separation in the final output.
Added support for Ckpt and Pt files.
More bug fixes and optimizations of course.
Previously:
Stylize mode added (vid2vid) DiffEx will automatically create prompts out of your videos. You can select them from your hard drive or automatically download them using yt-dlp (built in).
Added AnimateDiff and Mediapipe Face ControlNets.
Added the Refiner to upscale your animations.
Added the ability to restrict ContolNets to specific Regions.
Added Context Scheduling options for more stable animations.
Redesigned ControlNet image mapping interface.
Tooltips added to various controls to help newer users.
Regions! You can have up to 3 regions including the background (root prompt). This allows you to separate your animation into different layers allowing you greater control over the output.
The new MaskMaker allows you to automatically generate masks from videos for your regions.
LoRA mapping allows you to isolate LoRAs to a specific region and ramp its strength along the duration of your animation.
You can now paste prompt maps (ex. generated in ChatGPT) directly into the prompt grids.
QR Code Monster v1 and v2 added.
LCM support added.
SDXL support added.
Lots and lots of bug fixes and optimizations.
Features:
Automatically imports your Models, Motion Modules, VAEs, LoRAs and Embeddings.
IP Adapter support
Full multi ControlNet support.
Easily import video and images to create your ControlNet image map using drag and drop.
Diffex will automatically copy and sequence all of your images/videos and put them in their necessary folders.
Upcoming features:
adetailer integration to fix faces
EXIF data reader so you can import images you made in A1111 (and possibly Comfy)
Feel free to get in touch with me if you have any questions, suggestions or issues with the software.
Discord: https://discord.gg/ajzA9NVZCc
Reddit: https://www.reddit.com/user/tnil25
Description
See description for new features.
See included ReadMe for detailed install instructions, usage and common troubleshooting solutions.
Fixed an issue causing an index error when linking.
Fixed an issue with non-fatal errors causing DiffEx to stop generation.
Fixed a sync issue when using the Stylize source video in IP-Adapter.
Fixed an issue causing an error if no VAE directory is found.
Fixed an issue with spaces in file paths causing console errors.
Updated the installer syntax for better compatibility.
FAQ
Comments (82)
Great work, if only I could run it on 4GB.😭
Hi guahunyo, my discord name is: traxxas25. Feel free to reach out!
@tnil25 thankyou
@guahunyo No problem, I also set up a discord for DiffEx: https://discord.gg/5yWGTgVv
I'm gonna give it a shot :) nice idea !
Thank you! Let me know what you think!
@tnil25 its working fine awesome work, first suggestion could be a way to queue batch multiple random seed. But anyway i havent scratch the surface
Good job
@AsanaJM Good to hear! Ill absolutely consider adding a queue option (wrote it on my list), shouldn't be too difficult!
I followed the readme, but the exe doesn't seem to run. I'm not sure exactly what's happening as there is no error message or anything like that. :(
Hey Vokar, did you unzip all the files to the same folder? Make sure you have the .Net framework installed as well: https://dotnet.microsoft.com/en-us/download/dotnet-framework
I did, unzip the files, but I'll double check and do a couple of restarts to see if that helps.
Restarting worked for loading the app. Now I am receiving an 'onnxruntime' error.
@vokar28
In the readme, after entering venv\Scripts\activate.bat
Enter: python -m pip install -e .[dwpose]
Should fix your problem
@tnil25 I had to install the onnx by itself, but now I'm cruising! Thanks
@vokar28 Excellent! Enjoy.
Just tried it and gives errors sadly, does not work with samples provided either. I did all the steps in readme step by step with no issues.
You'll need to have the models that the sample json uses or it wont load correctly.
Model: https://civitai.com/models/4384/dreamshaper
Motion Module: https://huggingface.co/guoyww/animatediff/tree/main (I used v2 for the sample)
Hey everyone, if your having loading errors or LoRA path issues, download the new version: 1.2.5
DiffEx will now tell you what models it couldn't find and still load the rest of the Json.
There was a bug in the last repo that caused double \data\ to be added to the LoRA path. This is fixed now.
Hi, thanks for your great work.
But I have some trouble when I use the awesome software. I hope to get your help.
When I select the animatediff-cli-prompt-travel direction on my computer, there is always an error message "The selected folder doesn't appear to be valid"
Do we must use venv or may I need to install all the required dependences (pandas, ffmpeg)?
Or maybe I need to update the git commit to latest.
Hey, make sure to install the repo through CMD, not powershell. Follow the steps in the readme in the new 1.2.5 version.
@tnil25 Thank you, I will have a try. :)
I also cant link to CLI AD. It doesnt have to be in the root does it? Ive got mine in G:\SD installs\animatediff-cli-prompt-travel
did you ever figure this out? getting the same error
For those that are having trouble, please join our discord and message me so I can help you: https://discord.gg/ajzA9NVZCc
@TheValkyrie Please send me a message on discord so I can help you: https://discord.gg/ajzA9NVZCc
"Using DiffEx should be straight forward if you've ever made Json files for AnimateDiff. Again, c0nsumption's tutorial linked above will guide you through how it works. DiffEx automates a lot of this with a clean, easy to follow interface and performs the execution for you. There is no need to manually edit your Json file, or to go into the command line."
this part is not so good....you got to go into this with the assumption that the person using this has NEVER done anything with any of this.
I see what this does, it makes a json for you...but now i have no idea what to do with it.
hey omegablast, join our discord and ill be glad to assist you.
Ofcourse some prior knowledge of animatediff/stable diffusion will help. But its not required.
https://discord.gg/7NCaV6w8
Followed readme, animateDiff is installed correctly... However, linking fails: "The selected folder does not appear to be valid"
I'll hop on Discord later, but my assumption is that there is some hardcoded expectancy of C:\ or base drive directory, which would be silly if this was the case.
me too
same here
:)
It is likely the installation didn't complete successfully. Make sure to install Python and add it to PATH during installation. The size of the animatediff-cli-prompt-travel folder should be around 6gb in size.
If you're still having trouble join our discord and I can help.
Reuploaded an installer for the animatediff repo. Turns out distributing the repo wasn't possible. So I made a batch file that will perform the whole installation for you. Just download it to the folder you want to install to and run it.
@wyxzddsjj919 Haven't tried it yet, but Ill be sure to soon!
when will update?
As a MAC user, I was unable to use it, so I deleted it. I hope you develop a dmg file in the future.
Unix, ie, MacOS, has built in zip compression for 22 years.
I am not talking about zip compression. the program shows in .exe format instead of .dmg.
As a PC user , lol!!!!!
Maybe once I get the full feature set in place I can plan for a mac version. Have you tried using WINE to run it? Not sure if it will work but its worth a try.
Another Mac user here. Would be great if you were able to make this platform agnostic. Automatic1111 and ComfyUI both run on Windows, Mac, and Linux.
well can this do style swap? like I got the character design from bing . and then apply to video animation?
Sure, you can do that with IP-Adapter and ControlNet. Its all in there.
great for making batch openpose images, check out my page to get some few,
Am I supposed to be getting this?
(venv) C:\AnimateDiffPrompt\animatediff-cli-prompt-travel>animatediff
C:\AnimateDiffPrompt\animatediff-cli-prompt-travel\venv\lib\site-packages\controlnet_aux\segment_anything\modeling\tiny_vit_sam.py:654: UserWarning: Overwriting tiny_vit_5m_224 in registry with controlnet_aux.segment_anything.modeling.tiny_vit_sam.tiny_vit_5m_224. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
return register_model(fn_wrapper)
There are several of them on loading of the program, with tiny_vit_5m_#### being the only thing different in each.
Yes that's normal, you can ignore it.
It's cool, but I prefer portable version.
Is there a portable version?
Can I install it on a laptop then move the file to my desktop pc? (ie: does it install a portable version). Thank you for your help
You would need to run the installer again on your PC. But you can move all your models, outputs, etc to your PC.
AssertionError: Torch not compiled with CUDA enabled
Made sure CUDA was installed and reinstalled pytorch, I made sure python could see pytorch.
When I click execute and start the process this is the error I get.
Hey, there's a fix for this on our Discord. Requires downloading a bat file. Discord: https://discord.gg/ajzA9NVZCc
The Tile upscaling feature has it's own life.
It's starting when not enabled, I think when checking the box and unchecking it keeps it in the checked state?
It shouldn't. Are you using the latest version of DiffEx? Also, do you have the Console visible or no?
@tnil25 Yes v1.5.1. console is visible yes.
Sometimes if the console is visible it doesn't take the stop command that's supposed to be sent before upscaling begins. When the console is hidden it will be more responsive to the commands DiffEx sends it.
So try hiding the console and it should fix your issue. You can view console output in the Log.
Invalid value for 'STYLIZE_DIR': Directory 'C:\\MY' does not exist. getting this error on video to video conversion.
Check to make sure you are using the latest version of DiffEx. There was an issue before if the path had a space in it. It should be fixed now.
You make software with too many errors. not easy to use
What error are you receiving?
is their a Image to video mode ?
fix up the prblem.
i fail install it in anaconda python 3.10.14. it can not work with GPU.
UserWarning: Specified provider 'CUDAExecutionProvider' is not in available provider names.Available providers: 'AzureExecutionProvider, CPUExecutionProvider'
warnings.warn(
There is a fix for this in our discord, it requires downloading a bat file. Discord link is in the description.
Does this support AMD GPUs?
I haven't tested. I'm pretty sure it requires CUDA for GPU acceleration.
Do you have plans to support Chinese？or how can i change it on my own
A Linux user here. Would love to see this implemented with a cross-platform support. Another option would be to share the source code and let others help with the cross-platform implementation or maybe just to serve as an inspiration/base to create an OS-specific tool.
animediff folder is not accepted, do i need to rename the modelfiles somehow?
Can you send me a message on discord so I can help you with this issue? Discord link is in the description.
Please add Cpu Button or add directml or Zluda support but for now please add Cpu enable option in setting.u can add Cpu mode on execute button just add -d cpu after generate word I'm looking through the files to see if I can find the execute button code I can't find it anywhere in it has to be in the diffex folder somewhere
I keep getting this error while installing from scratch:
ModuleNotFoundError: No module named 'torch' [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error × Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> See above for output. note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: triton-2.0.0-cp310-cp310-win_amd64.whl is not a supported wheel on this platform.
Details
Files
Available On (1 platform)
Same model published on other platforms. May have additional downloads or version variants.