This workflow will take any prompt and significantly enhance it using an LLM to provide a very detailed prompt designed specifically for T5 natural language prompting (Flux, SD3).
This workflow was only possible for me by the impressive work of Reddit user u/glibsonoran for creating the Plush-for-ComfyUI nodes and u/Relative_Bit_7250 for his talented work creating the Custom GPT "Image Prompt Generator" which heavily guided my LLM instructions here.
You can use this with a variety of LLMS, but I recommend Groq simply because you can get a free API key that will do all you need for this workflow. Pay particular attention to setting up the API key environment variable, see here: https://github.com/glibsonoran/Plush-for-ComfyUI#requirements
Plush-for-ComfyUI: https://github.com/glibsonoran/Plush-for-ComfyUI
GPT Image Prompt Generator: https://chatgpt.com/g/g-OndbVJ3Et-image-prompt-generator
Description
FAQ
Comments (7)
How do i implement a local llm?
There is a place in the node to enter your local LLM http address. However, I recommend Groq since it's entirely free and works great. If you want to read more about using a local LLM read through the recent updates here: https://github.com/glibsonoran/Plush-for-ComfyUI#updates
@Backroads4me thanx, but i don´t understand it... :-(
running local will also use up a good portion of your VRAM. Either you wont be getting good prompts, or you wont be generating images as good as you could be. You will likely have to juggle loading and unloading your llm every time you want to use it. I guess you could use a lamaacpp or a koboldcpp and offload it all to your cpu, but that would be real slow and limit you to a gguf model. not worth the rabbit hole you'd have to go down.
AFTER setting my "GROQ_API" key I was getting a message "Server unable to complete the request". I tried refreshing, nothing. I couldn't select a GROQ_Model in the LLM Enhanced Node as the options were <NONE>.
RESTARTING Comfyui fixed this. It works. :)
RECOMMAND 2 EXTs with llm and llm-vision
Quick Links
Auto prompt by LLM and LLM-Vision (Trigger more details out inside model)
SD-WEB-UI: https://github.com/xlinx/sd-webui-decadetw-auto-prompt-llm
ComfyUI: https://github.com/xlinx/ComfyUI-decadetw-auto-prompt-llm
Auto msg to ur mobile (LINE | Telegram | Discord)
SD-WEB-UI :https://github.com/xlinx/sd-webui-decadetw-auto-messaging-realtime
ComfyUI: https://github.com/xlinx/ComfyUI-decadetw-auto-messaging-realtime
Hello. When I try to load your workflow, I get the message that I'm missing the following node type:
"UNETLoaderMultiGPU"
Can't find info on this.