This is a new VAE for SDXL which outputs images in the Rec. 2100 PQ color space in order to allow for HDR images. This should be used together with a model or LORA which takes advantage of it.
Important:
This will not create correct images out-of-the-box, as it will be interpreted as being in a wrong color space. You need to manually change the color space of the images after generation, otherwise images will look gray and boring.
Currently only Chromium based browsers support HDR and you will need to have a monitor that supports HDR and have it enabled in order to get any HDR effect.
HDR requires a higher bit depth and you can get posterization artifacts in 24-bit color. If you are making SDR images, you should use the normal VAE for better quality.
Rationale:
LORAs can be used to change the style of a model, but while it can approximate a new color space this way, it will not necessarily map every concept it knows correctly. A prompt specifying colors such as "red dress" can cause it to create the wrong color as it tries to use its old representation of it. This VAE is trained to keep the original latent representation as it has always been, and instead map the "red" color to the new color space in the VAE. This is way more robust than trying to handle all edge cases in the LORA training.
To support HDR, a LORA is trained just to push the generated latent outside the normal SDR range for concepts directly related to HDR.
Tools:
This VAE contains a new meta-data entry to specify its color space:
'modelspec.color_space': 'cicp:9,16,0,true',This is not yet standardized and image generators will incorrectly assume the image is in sRGB. To create correct looking images, you need to manually fix the images.
To override the color space for PNGs, I have created a python script you can run here:
https://github.com/spillerrec/rec2100-vae-training/blob/master/mark-png-as-hdr.py
This will produce grayish images on browsers which does not support HDR, but is currently the easiest solution right now.
A better approach is to use UltraHDR which is an extension to JPG. This has better backwards compatibility and will show a normal SDR image in most applications, and show HDR in supported ones. I will try to create a way to do this in the near future.