QR Code Conditioned ControlNet Models for Stable Diffusion 1.5 and 2.1

@diontimmer That's not the point, I can generate readable codes with 1.2-1.5 weight. The problem is that the code itself is too noticeable. Compare it with the images that started it all: https://www.reddit.com/r/StableDiffusion/comments/141hg9x/controlnet_for_qr_code/ Only the 3 big squares and partially the little alignment pattern in the bottom right are perfectly visible, the rest blends with the image nicely. This is exactly what gives the "wow" effect because no one expects this image to be a scannable code and yet it is one.

I suppose the QR scan software doesn't look for big data blocks, only the position blocks and for data it probably reads just 1 pixel at the expected location. The big rectangles let it calculate the orientation, position and perspective distortion and compensate for it, then it just needs to pick one central pixel for eact data square. If ControlNet exploits this fact (or rather, guess) it'd be able to leave most of the image intact and only brighten/darken specific small points + you always have those 30% error correction codes in case it's not perfect.

I wonder if CN is capable of doing this preprocessing for us. I read a bit about training it, I don't know how you did it but I suppose you prepared 100k QRs, 100k images and blended the QRs on top of the images using one of the standard photoshop blend options, whatever gives the better result. So what if you did the following:

1) Make QR pairs, they should encode the exact same data but represent it differently. QR 1 is the usual QR you can generate anywhere. QR 2 has the data squares as small as possible, similar to https://uniqr.us/ but on transparent background (should be achievable with custom renderer, there are tons of QR libs)

2) Overlay QR 2 on top of your image dataset using the same blending you already did, white dots brighted the pixel/small rectangle, black dots darken it, transparent background doesn't change anything

3) Use QR 1 as the conditioning image

That way people would use regular unprocessed QRs but SD would output images with only small dots highlighted/darkened and big squares in the corners still being big. So the code would be readable and the image would look like a normal image and not a QR on top of it. I'm really curious if it works out, might try it myself even, lol.

Ah nvm, I can get some interesting result with custom QR codes that use non-square dots. They still should be quite large but if I choose something angular or round they can better hide in the image.

I think the never released model was that good because the devs trained it for a very long time on a very powerful hardware according to their article, so it learned to blend the QR with the image extremely well. Don't get me wrong, your model is also awesome and with a clever QR design and prompt it's possible to make interesting images too. Thank you for your work!

no one cares about apple's shit
get outta here

@dominux Being poor is not a reason to be mean.