Feedback is welcome! I'm still improving this model and you can help me by generating <a target="_blank" rel="ugc" href="https://civarchive.com/api/download/attachments/185">simple usecases</a> and discuss the results. ControlNet training alone is enough work for one person already :)Update: I created a <a rel="ugc" href="https://civarchive.com/bounties/28">buzz bounty</a> for using my ControlNet!<h1 id="heading-2385">Summary</h1>Generate images which are controlled for light and dark areas by using white-gray-black control images. I didn't want to create yet another QR Code model to compete with Monster Labs model which is already great. My idea was to focus more on natural and organic images for squint illusions as opposed to QR codes and take more regard to human perception as opposed to how cameras interpret contrast. You should notice a white color in the condition image become more of a light-color (HSV) instead of THE white-color (RGB). I also found the Monster Labs model tends to produce images with higher contrast, probably to make it work better with QR codes. While this model is able to produce scannable QR codes it was not the goal.<h1 id="heading-2387">Usage</h1>Use it by setting <code>strength=0.6-0.8</code> and <code>start_percent=0-0.1</code>, <code>end_percent=0.6-1.0</code> depending on your image concept. I generally found that you have to use lower ControlNet strength compared to other QR code models. Make a XYZ-plot with different start, end and weight strengths to get an idea how it influences the generation. You also don't have to use white-gray-black images but can use grayscale or RGB images as well. I also found that fine-tuned models produce more artistic results.<h1 id="heading-231">Templates</h1>I have made a <a target="_blank" rel="ugc" href="https://civarchive.com/api/download/attachments/185">collection of different templates</a> which you can download to get started:<img src="https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/97fb48ab-e2f5-44c0-9a06-a76ed96e4432/width=525/97fb48ab-e2f5-44c0-9a06-a76ed96e4432.jpeg" /><a target="_blank" rel="ugc" href="https://civarchive.com/api/download/attachments/185">Download here!</a><h1 id="heading-2386">Training</h1>See my article <a target="_blank" rel="ugc" href="https://civarchive.com/articles/2078">Play in Control - ControlNet Training Setup Guide</a> if you want to train your own control net!The model was trained on the laion2b-en-aesthetics65 by converting images to grayscale using HSV method, Gaussian blur with kernel=15 and remapping colors to 33% white-gray-black. My idea was to imagine existing images as having been generated by a white-gray-black pattern image.<img src="https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/cc003369-900e-46ff-b6d9-d62b187f2895/width=525/cc003369-900e-46ff-b6d9-d62b187f2895.jpeg" /><h2 id="heading-470">Versions</h2><ul><li>v0.4: 210000a32 samples with mixed_precision="fp16"</li><li>v0.3: 130000a4 samples with mixed_precision="fp16"</li><li>v0.2: 180000a4 samples with mixed_precision="fp16"</li><li>v0.1: 68000a4 samples with mixed_precision="no"</li></ul>I will improve and train new versions based on usage and feedback.<h1 id="heading-2393">References</h1><h2 id="heading-2394">Tutorials</h2><a target="_blank" rel="ugc" href="https://antfu.me/posts/ai-qrcode-refine">Antfu - Artistic QR codes</a><a target="_blank" rel="ugc" href="https://stable-diffusion-art.com/qr-code">Saigo - Artistic QR codes</a><a target="_blank" rel="ugc" href="https://takin.ai/learn/generate-images-with-hidden-text-using-stable-diffusion-and-controlnet">Takin - Hidden text</a><h2 id="heading-2395">Inspirations</h2><a target="_blank" rel="ugc" href="https://www.robgonsalves.live/gallery">Rob Gonsalves</a><a target="_blank" rel="ugc" href="https://www.reditr.com/r/StableDiffusion/comments/16ew9fz/spiral_town_different_approach_to_qr_monster">reddit - Spiral Town</a><a target="_blank" rel="ugc" href="https://www.reddit.com/r/StableDiffusion/comments/16bqxmp/what_do_you_see_part_i">reddit - Hidden logos</a><a target="_blank" rel="ugc" href="https://www.reditr.com/r/StableDiffusion/comments/16ee6lf/what_do_you_see_part_ii">reddit - Silhoutte illusions 1</a><a target="_blank" rel="ugc" href="https://www.reddit.com/r/StableDiffusion/comments/15xh4yi/more_of_these_optical_illusions">reddit - Silhoutte illusions 2</a><a target="_blank" rel="ugc" href="https://www.reddit.com/r/StableDiffusion/comments/167kwcb/portraits_with_notsosubliminal_messages">reddit - Subliminal symbols</a><a target="_blank" rel="ugc" href="https://civarchive.com/user/antfu">antfu - Stylized QR codes</a><a target="_blank" rel="ugc" href="https://qrbtf.com/gallery">QRBTF (gallery and generator)</a><h2 id="heading-2396">Tools</h2><a target="_blank" rel="ugc" href="https://qrcode.antfu.me">QR code generator and verifier</a><a target="_blank" rel="ugc" href="https://github.com/antfu/sd-webui-qrcode-toolkit">Automatic1111 QR integration</a><h2 id="heading-2397">Similar models</h2><a target="_blank" rel="ugc" href="https://civarchive.com/models/111006/qr-code-monster">https://civarchive.com/models/111006/qr-code-monster</a><a target="_blank" rel="ugc" href="https://civarchive.com/models/90940/controlnet-qr-pattern-qr-codes">https://civarchive.com/models/90940/controlnet-qr-pattern-qr-codes</a><a target="_blank" rel="ugc" href="https://civarchive.com/models/80536/lighting-based-picture-control-controlnet">https://civarchive.com/models/80536/lighting-based-picture-control-controlnet</a><a target="_blank" rel="ugc" href="https://civarchive.com/models/90472/controlnet-qr-code">https://civarchive.com/models/90472/controlnet-qr-code</a>Thanks to antfu for the great in-depth articles and Monster Lab for the QR Code Monster model and the art generated henceforth inspiring me to get into control net training myself!

ControlNet Mysee - Light and Dark - Squint Illusions Hidden Symbols Subliminal Text QR Codes

New version has more freedom to generate creative interpretations of light and dark areas.New version should be compatible with Automatic1111.Trained with HuggingFace diffusers <code>train_controlnet.py</code> with:<pre><code>accelerate launch train_controlnet.py ^
 --pretrained_model_name_or_path="runwayml/stable-diffusion-v1-5" ^
 --mixed_precision="fp16" ^
 --resolution=512 ^
 --learning_rate=1e-5 ^
 --train_batch_size=1 ^
 --gradient_accumulation_steps=4 ^
 --gradient_checkpointing ^
 --use_8bit_adam ^
 --enable_xformers_memory_efficient_attention ^
 --set_grads_to_none ^
 --seed=0</code></pre>on 32500 steps (x4 = 130000 images) from <code>laion2B-en-aesthetics65</code> converted with:<pre><code>magick convert xc:black xc:#808080 xc:white +append palette.png
magick convert input.webp -resize 64x64 -filter Lanczos +dither -remap palette.png -resize 512x512 -filter point -define webp:lossless=false output.png</code></pre>

Files with SHA256: c71f47b6586d02597f7cf11a6458b7d1e99162d03e21411f7675e746af39b36e