<h1 id="flux-1.dev.fp8-+-controlnet-canny-11sqfvpov">Flux-1.dev.fp8 + ControlNet Canny</h1><s>At the time of writing this guide, it is possible to try out controlnet canny only by switching your ComfyUI from master to xlabs_flux_controlnet (git branch).</s>Let's jump into the prerequisites: Prerequisites<ul><li>Fetch and update all custom nodes</li><li>Update ComfyUI from the ComfyUI manager</li><li>Install Missing Custom Nodes from ComfyUI manager and Restart it</li></ul>HOW DOES IT WORKRequisites<ul><li>download UNET <code>flux-dev-fp8</code> <a target="_blank" rel="ugc" href="https://huggingface.co/Comfy-Org/flux1-dev/blob/main/flux1-dev-fp8.safetensors">https://huggingface.co/Comfy-Org/flux1-dev/blob/main/flux1-dev-fp8.safetensors</a>, save into <code>models/unet</code> (only this one work, not <a target="_blank" rel="ugc" href="http://xlabs.ai">xlabs.ai</a> or others)</li><li>download VAE <code>ae</code> model <a target="_blank" rel="ugc" href="https://huggingface.co/black-forest-labs/FLUX.1-schnell/blob/main/ae.safetensors">https://huggingface.co/black-forest-labs/FLUX.1-schnell/blob/main/ae.safetensors</a> save it into <code>models/vae</code></li><li>download CLIPS: <code>t5xxl_fp16.safetensors</code> <code>clip_l.safetensors</code></li></ul>(optional if your machine has less than 32GB of TvT ram) <code>t5xxl_fp8_e4m3fn.safetensors</code>at: <a target="_blank" rel="ugc" href="https://huggingface.co/comfyanonymous/flux_text_encoders/tree/main">https://huggingface.co/comfyanonymous/flux_text_encoders/tree/main</a>save them into <code>ComfyUI/models/clip/</code>1 UNET + CLIP - Group<ul><li>1 we load flux1-dev-fp8.safetensors, the only one working as of today (<a target="_blank" rel="ugc" href="http://Comfy.org">Comfy.org</a>) <a target="_blank" rel="ugc" href="https://huggingface.co/Comfy-Org/flux1-dev/blob/main/flux1-dev-fp8.safetensors">https://huggingface.co/Comfy-Org/flux1-dev/blob/main/flux1-dev-fp8.safetensors</a> and the dual clip loader for Flux-1.</li><li>2 select 1 for unet only, select 2 for unet + flux_realism_lora</li><li>3 we set some globals to avoid node splines cluttering via SetNode.</li></ul>2 Base Image - Group<ul><li>1 Load your base image</li></ul>3 Prompts - Group<ul><li>1 Image to text via Florence2</li><li>2 concatenate Florence caption Generic prompt via String Function and pass it to conditioning global PROMPT_POS-COND</li></ul>4 CNET - Group (Controlnet)<ul><li>select 1 for prompt only OR select 2 for prompt + canny-controlnet </li></ul>A few NotesSeed 688106326432651Cnet 0.76 head facing the camera , still nice imageCnet 0.77 pulls up her head, still nice imageCnet 78 the cnet kick in violently and fuck up her armsLora:Keep it low, it's very strongA Big shoutout to <a target="_blank" rel="ugc" href="https://www.reddit.com/user/comfyanoymus/">u/comfyanoymus</a> and the amazing <a target="_blank" rel="ugc" href="http://Comfy.org">Comfy.org</a> project/community

Files with SHA256: 514adf69402f87e60f616a0a7ebfcef6ce84847a93af40a96983e533f18273c0