<h1>sd-webui-bayesian-merger</h1><h2>What is this?</h2><p>An opinionated take on stable-diffusion models-merging automatic-optimisation.</p><p>The main idea is to treat models-merging procedure as a black-box model with 26 parameters: one for each block plus <code>base_alpha</code> (note that for the moment <code>clip_skip</code> is set to <code>0</code>).</p><p>We can then try to apply black-box optimisation techniques, in particular we focus on <a rel="ugc" href="https://en.wikipedia.org/wiki/Bayesian_optimization">Bayesian optimisation</a> with a <a rel="ugc" href="https://en.wikipedia.org/wiki/Gaussian_process">Gaussian Process</a> emulator.</p><p>Read more <a rel="ugc" href="https://github.com/fmfn/BayesianOptimization">here</a>, <a rel="ugc" href="http://gaussianprocess.org">here</a> and <a rel="ugc" href="https://optimization.cbe.cornell.edu/index.php?title=Bayesian_optimization">here</a>.</p><p>The optimisation process is split in two phases:</p><p>1. <strong>exploration</strong>: here we sample (at random for now, with some heuristic in the future) the 26-parameter hyperspace, our block-weights. The number of samples is set by the</p><p><code>--init_points</code> argument. We use each set of weights to merge the two models we use the merged model to generate <code>batch_size * number of payloads</code> images which are then scored.</p><p>2. <strong>exploitation</strong>: based on the exploratory phase, the optimiser makes an idea of where (i.e. which set of weights) the optimal merge is.</p><p>This information is used to sample more set of weights <code>--n_iters</code> number of times. This time we don't sample all of them in one go. Instead, we sample once, merge the models,</p><p>generate and score the images and update the optimiser knowledge about the merging space. This way the optimiser can adapt the strategy step-by-step.</p><p>At the end of the exploitation phase, the set of weights scoring the highest score are deemed to be the optimal ones.</p><h3>Juicy features</h3><p>- wildcards support</p><p>- TPE or Bayesian Optimisers. cf. <a rel="ugc" href="http://papers.neurips.cc/paper/4443-algorithms-for-hyper-parameter-optimization.pdf">Bergstra et al. 2011</a> for a comparison</p><p>- UNET visualiser</p><p>- convergence plot</p><h2>OK, How Do I Use It In Practice?</h2><p>Head to the <a rel="ugc" href="https://github.com/s1dlx/sd-webui-bayesian-merger/wiki/Home">wiki</a> for all the instructions to get you started.</p><p></p>

Files with SHA256: ef6f53aaae81ab7068aaf8ac74ae1a64f7569766fb6cd5a7f2a5ccc664addb72