<h3 id="controlnetflux-(cnflux)-a-collection-of-controlnet-models-for-flux.1-dev-dbc3x7t3x">ControlNetFlux (CNFlux) - A collection of Controlnet models for Flux.1 [dev]</h3>This collection strives to create a convenient download location of all currently available Controlnet models for Flux. Please do read the version info for model specific instructions and further resources. If you are the author of one of these models and don't want it to appear here, please contact me to sort this out. Support at the moment is limited to ComfyUI and Forge as far as i know.NEW: windsingai-openposeA ComfyUI workflow for InstantX's union model can be found <a target="_blank" rel="ugc" href="https://www.reddit.com/r/StableDiffusion/comments/1euz2a9/union_flux_controlnet_running_on_comfyui_workflow/">here</a>. The necessary nodes <a target="_blank" rel="ugc" href="https://github.com/EeroHeikkinen/ComfyUI-eesahesNodes">here</a>.A huge thanks to all the authors, devs and contributors including but not limited to: Automatic1111, Black-Forest-Labs, comfyanonymous, dashtoon, eesahe, Illasviel, InstantX, kadirnar, lucataco, PromeAI, Shakker-Labs, windsingai, <a target="_blank" rel="ugc" href="https://civarchive.com/user/xlabs_ai">XLabs-AI</a>

ControlNetFlux (CNFlux)

Inpainting Beta model released by AlimamaCreative on huggingface<a target="_blank" rel="ugc" href="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Beta">https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Beta</a>

controlnetfluxCnflux_alimamaInpaint-inpainting.safetensors

alimama-inpaintβ

windsingai-openpose

raulc0399-openpose

george0667-linecombo

jasperai-upscale

jasperai-depth

jasperai-normal

lucataco-fill10k

lucataco-qrcode

dashtoon-scribble

promeai-lineart

shakkerlabs-depth

shakkerlabs-union-pro

themistoai-mistoline

kadirnar-canny-16k

xlabsai-ipadapter

instantx-union

xlabsai-hed-v3

xlabsai-depth-v3

xlabsai-canny-v3

instantx-union-α

instantx-canny

xlabsai-hed

xlabsai-depth-v2

xlabsai-canny-v2

This repository provides a Inpainting ControlNet checkpoint for <a target="_blank" rel="ugc" href="https://huggingface.co/black-forest-labs/FLUX.1-dev">FLUX.1-dev</a> model released by AlimamaCreative Team.<h2 id="news-4ddurt2ar">News</h2>🎉 Thanks to @comfyanonymous，ComfyUI now supports inference for Alimama inpainting ControlNet. Workflow can be downloaded from <a target="_blank" rel="ugc" href="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/alimama-flux-controlnet-inpaint.json">here</a>.ComfyUI Usage Tips:<ul><li>Using the <code>t5xxl-FP16</code> and <code>flux1-dev-fp8</code> models for 28-step inference, the GPU memory usage is 27GB. The inference time with <code>cfg=3.5</code> is 27 seconds, while without <code>cfg=1</code> it is 15 seconds. <code>Hyper-FLUX-lora</code> can be used to accelerate inference.</li><li>You can try adjusting（lower） the parameters <code>control-strength</code>, <code>control-end-percent</code>, and <code>cfg</code> to achieve better results.</li><li>The following example uses <code>control-strength</code> = 0.9 &amp; <code>control-end-percent</code> = 1.0 &amp; <code>cfg</code> = 3.5</li></ul>InputOutputPrompt<a target="_blank" rel="ugc" href="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/comfy_in_1.png"><img src="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/comfy_in_1.png" alt="Image1" /></a><a target="_blank" rel="ugc" href="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/comfy_out_1.png"><img src="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/comfy_out_1.png" alt="Image2" /></a>The image depicts a scene from the anime series Dragon Ball Z, with the characters Goku, Elon Musk, and a child version of Gohan sharing a meal of ramen noodles. They are all sitting around a dining table, with Goku and Gohan on one side and Naruto on the other. They are all holding chopsticks and eating the noodles. The table is set with bowls of ramen, cups, and bowls of drinks. The arrangement of the characters and the food creates a sense of camaraderie and shared enjoyment of the meal.<a target="_blank" rel="ugc" href="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/comfy_in_2.png"><img src="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/comfy_in_2.png" alt="Image3" /></a><a target="_blank" rel="ugc" href="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/comfy_out_2.png"><img src="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/comfy_out_2.png" alt="Image4" /></a>The image is an illustration of a man standing in a cafe. He is wearing a white turtleneck, a camel-colored trench coat, and brown shoes. He is holding a cell phone and appears to be looking at it. There is a small table with a cat on it to his right. In the background, there is another man sitting at a table with a laptop. The man is wearing a black turtleneck and a tie.<a target="_blank" rel="ugc" href="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/comfy_in_3.png"><img src="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/comfy_in_3.png" alt="Image5" /></a><a target="_blank" rel="ugc" href="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/comfy_out_3.png"><img src="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/comfy_out_3.png" alt="Image6" /></a>A woman with blonde hair is sitting on a table wearing a red and white long dress. She is holding a green phone in her hand and appears to be taking a photo. There is a bag next to her on the table and a handbag beside her on the chair. The woman is looking at the phone with a smile on her face. The background includes a TV on the left wall and a couch on the right. A chair is also present in the scene.<a target="_blank" rel="ugc" href="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/comfy_in_4.png"><img src="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/comfy_in_4.png" alt="Image7" /></a><a target="_blank" rel="ugc" href="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/comfy_out_4.png"><img src="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/comfy_out_4.png" alt="Image8" /></a>The image depicts a beautiful young woman sitting at a desk, reading a book. She has long, wavy brown hair and is wearing a grey shirt with a black cardigan. She is holding a red pencil in her left hand and appears to be deep in thought. Surrounding her are numerous books, some stacked on the desk and others placed on a shelf behind her. A potted plant is also visible in the background, adding a touch of greenery to the scene. The image conveys a sense of serenity and intellectual pursuits.<h2 id="model-cards-ho3gz3ov5">Model Cards</h2><ul><li>The model was trained on 12M laion2B and internal source images at resolution 768x768. The inference performs best at this size, with other sizes yielding suboptimal results.</li><li>The recommended controlnet_conditioning_scale is 0.9 - 0.95.</li><li>Please note: This is only the alpha version during the training process. We will release an updated version when we feel ready.</li></ul><h2 id="showcase-5vjqhgjwh">Showcase</h2><a target="_blank" rel="ugc" href="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/blob/main/images/flux1.jpg"><img src="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/flux1.jpg" alt="flux1" /></a><a target="_blank" rel="ugc" href="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/blob/main/images/flux2.jpg"><img src="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/flux2.jpg" alt="flux2" /></a><a target="_blank" rel="ugc" href="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/blob/main/images/flux3.jpg"><img src="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/flux3.jpg" alt="flux3" /></a><h2 id="comparison-with-sdxl-inpainting-jri6729g4">Comparison with SDXL-Inpainting</h2>Compared with <a target="_blank" rel="ugc" href="https://huggingface.co/diffusers/stable-diffusion-xl-1.0-inpainting-0.1">SDXL-Inpainting</a>From left to right: Input image | Masked image | SDXL inpainting | Ours<a target="_blank" rel="ugc" href="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/blob/main/images/0.jpg"><img src="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/0.jpg" alt="0" /></a>The image depicts a beautiful young woman sitting at a desk, reading a book. She has long, wavy brown hair and is wearing a grey shirt with a black cardigan. She is holding a pencil in her left hand and appears to be deep in thought. Surrounding her are numerous books, some stacked on the desk and others placed on a shelf behind her. A potted plant is also visible in the background, adding a touch of greenery to the scene. The image conveys a sense of serenity and intellectual pursuits.<a target="_blank" rel="ugc" href="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/blob/main/images/1.jpg"><img src="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/1.jpg" alt="0" /></a>A woman with blonde hair is sitting on a table wearing a blue and white long dress. She is holding a green phone in her hand and appears to be taking a photo. There is a bag next to her on the table and a handbag beside her on the chair. The woman is looking at the phone with a smile on her face. The background includes a TV on the left wall and a couch on the right. A chair is also present in the scene.<a target="_blank" rel="ugc" href="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/blob/main/images/2.jpg"><img src="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/2.jpg" alt="0" /></a>The image is an illustration of a man standing in a cafe. He is wearing a white turtleneck, a camel-colored trench coat, and brown shoes. He is holding a cell phone and appears to be looking at it. There is a small table with a cup of coffee on it to his right. In the background, there is another man sitting at a table with a laptop. The man is wearing a black turtleneck and a tie. There are several cups and a cake on the table in the background. The man sitting at the table appears to be typing on the laptop.<a target="_blank" rel="ugc" href="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/blob/main/images/3.jpg"><img src="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/3.jpg" alt="0" /></a>The image depicts a scene from the anime series Dragon Ball Z, with the characters Goku, Naruto, and a child version of Gohan sharing a meal of ramen noodles. They are all sitting around a dining table, with Goku and Gohan on one side and Naruto on the other. They are all holding chopsticks and eating the noodles. The table is set with bowls of ramen, cups, and bowls of drinks. The arrangement of the characters and the food creates a sense of camaraderie and shared enjoyment of the meal.<h2 id="using-with-diffusers-yg9jlelym">Using with Diffusers</h2>Step1: install diffusers<pre><code>pip install diffusers==0.30.2
</code></pre>Step2: clone repo from github<pre><code>git clone https://github.com/alimama-creative/FLUX-Controlnet-Inpainting.git
</code></pre>Step3: modify the image_path, mask_path, prompt and run<pre><code>python main.py
</code></pre><h2 id="license-33pdro9ex">LICENSE</h2>Our weights fall under the <a target="_blank" rel="ugc" href="https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md">FLUX.1 [dev]</a> Non-Commercial License.

alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha

alimamaCreativeFLUX1Dev_v10.safetensors

v1.0

Files with SHA256: 0e9d4891bb416584228b8fe16f35154ee462870f3798c108eef199d826c63ee2