Please update to latest Comfyui to apply this model! use ControlNetApply SD3 and HunyuanDiT node to apply the controlnet
Tencent HunyuanDiT Inpainting Controlnet
An amazing controlnet model which can provide you incrediable consistent and crativity to the original image!
it is trained with various resolution of images and masks to provide nice robust to the the base model. now it can create the new image with the same style of the original one, following are some example to show the result!
it can adapter to 6X-8X times fix from the original size of image. please do preprecess your image if your upscale rate is not drop in 6-8X
Preprocessor is here:
https://github.com/TTPlanetPig/Comfyui_TTP_CN_Preprocessor
for hunyuan model safetensors pack with mt5 download here:
https://huggingface.co/comfyanonymous/hunyuan_dit_comfyui/blob/main/hunyuan_dit_1.2.safetensors
Ultimate SD upscale will bring you even crazy result once the comfyui or webui is supported.
I upscale directly by linear for the images on left, and right side is the tile output
example1:
example2:
example3:
example4:
example5:
Other capablity!
NSFW is ready! once HunyuanDiT NSFW FT model is ready!
with current defualt model, you can even tile the nsfw content for most condition, however a ft model will be better to apply this controlnet model.
The end and enjoy
Join my QQ group if you appreiate my job: 294060503 or contact my wechat: tangtuanzhu
Description
formal version for Hunyuan 1.2
FAQ
Comments (10)
C'est incroyable!! Amazing!!
这么快就炼制出tile了
Is it possible for you to pass the model through Kohya or similar to swap the format to Safetensor?
I've renamed the model to "pytorch_model_tile_ema.pt" but I have errors.
"Error(s) in loading state_dict for HunYuanControlNet: Missing key(s) in state_dict:"
I also have warnings about "using a model of type mt5 to instantiate a model of type t5"
I'm on Windows.
Someone have an idea about what could be going wrong ?
remember modify the ./hydit/config.py change the line 50 to parser.add_argument("--control-type", type=str, default='canny', choices=['canny', 'depth', 'pose', 'tile'], help="Controlnet condition type")
one more reason is, the model is packed in the module shell, need to take it out to adapter the hunyuan code.
I have reported to Hunyuan ,and they fixed it.
they are updating the standard comfyui controlnet now, not pipeline version. so we can apply it to more interesting workflow!
tile需要搭配生成图像尺寸放大使用，但是dit的训练加入了RoPE会限定图片尺寸大小，这里不会影响tile方法图片尺寸吗
将来需要粉块方法，等comfyui支持了会做流程，类似SD3. 目前是8倍能力
It doesn't work, it is getting a K-Sampler error.
maybe you need to update the comfyui?