Changelogs on bottom of description. TLDR, V1 is better than V1.3 (test) Eclipse XL v1.0 is a fine-tuned model trained on 63K images, aimed at creating a higher-fidelity base Anime XL model. This was a collaborative effort by <a target="_blank" rel="ugc" href="https://civarchive.com/user/Wasabiya">Wasabi</a> and <a target="_blank" rel="ugc" href="https://civarchive.com/user/Hecatonchireart">Hecatonchirea</a>. We tackled this project with a token-based approach, using a dataset primarily consisting of Booru-based tags, along with a few additional Rule34 tags and original tags. The tags were pruned and cleaned using our tag-editing application (HW tagger, Link to <a target="_blank" rel="ugc" href="https://github.com/HaW-Tagger/HWtagger">github</a> and <a target="_blank" rel="ugc" href="https://civarchive.com/articles/5752/hw-tagger-application-how-to-use-basic-tutorial">tutorial</a>), with semi-manual inspection.Technically, the base model is Pony v6, so natural language prompting and Pony v6-based Loras will likely work with this model (although we don't recommend using Pony's quality tags, for reasons explained in the technical section). We introduced new tags for lighting (composition tags), new quality tags, and various other features to achieve better control over our generations. Our focus was not on characters or styles, as people will create Loras anyway, and using Loras will produce better images.We had many subgoals for the project, such as improving lighting, enhancing sensitivity to tags, overwriting the knowledge of Pony, separating the style tied to tags, achieving a consistent and flexible style, and preventing it from being style-hungry like Pony (more details in the technical section). We will provide more in-depth details in the technical article.We operate without any funding or sponsors, so if you appreciate the model, any amount of tips would be highly appreciated. You can also support us through our Patreon.This versatile model is capable of generating both SFW and NSFW images. Please use it responsibly. If you are unable to run XL models and haven't heard of SD Forge, I highly recommend looking it up, as it may help you run XL models more efficiently. We also recommend checking out the related article because it contains the csv for the tags used in this model which you can drop it into your webui tag-autocomplete extension.<h2 id="how-to-use:-j4ekhmsff">How to use:</h2>Recommended starter prompt:Positive prompt [prompt like any other tag based model]:<pre><code>masterpiece, best, great, ...</code></pre>Negative prompt (NO NEED for a long negative):<pre><code>worst, worse, average, signature, watermark</code></pre><h3 id="recommended-resolutions-and-prompting-method:-i9cwdyh4a">recommended resolutions and prompting method:</h3><pre><code>(768, 1280), 3:5 ratio
(768, 1344), 4:7 ratio
(832, 1216), 13:19 ratio
(896, 1152), 7:9 ratio
(960, 1088), 15:17 ratio
(1024, 1024), square ratio</code></pre><h3 id="we-recommend-cfg-between-5-8-sampling-steps-above-20-(we-use-36-steps)-and-clip-skip-1-(pony-says-to-use-2-but-clip-skip-is-disabled-for-xl-training-on-kohya-so-it-doesn't-make-sense-to-use-2-people-are-probably-gaslighting-themselves-from-sd-1.5-experience).-8p80w680c">We recommend CFG between 5-8, sampling steps above 20 (we use 36 steps), and CLIP SKIP 1 (Pony Says to use 2, but Clip Skip is disabled for XL training on kohya so it doesn't make sense to use 2, people are probably gaslighting themselves from SD 1.5 experience).</h3><h2 id="special-tag-info:-dbjrci6gr">Special tag info:</h2>We introduced new tags based on their understood meaning (or the lack there of) by the text encoder model (Vit and BIG) that is in the XL model. Most new tokens are 1~2 token long, so the information is better absorbed in the training.<h2 id="quality-tags:-izznzjdv1">quality tags:</h2><pre><code>masterpiece, best, great, good, average, worse, worst</code></pre>The quality tags were assigned using the aesthetic scorer from imgutils, we're aware of it's many biases so we manually corrected them. Although flawed, it's better than other options, so we ran with this, more info on the technical detail article.<h2 id="additional-detail-tags:-68a68d52c">additional detail tags:</h2><pre><code>dense, intricate</code></pre>These tags were added for images with many details or parts, some images had both:<ul><li>intricate : the details on objects/subject is tightly packed and is not a simple design (ex: lingerie, complex dress, designed armor trims, multiple accessories, etc)</li><li>dense : image that has multiple objects/subjects that makes the image more densely packed</li></ul><h2 id="lighting-tags:-lvjlm402o">lighting tags:</h2><pre><code>dim composition, ambient composition, dun composition, dark composition, contrast composition, bright composition, vibrant composition, dark background</code></pre>These are not necessary for basic lighting but these tags were added to images with very extreme lighting or darkness. We followed the definitions below for tagging the images for specific lighting scenarios so we're consistent with our tagging for the training, but you can mix them in generation to get interesting effects:<ul><li>dim composition : Dark fully visible image, but there are multiple sources of light</li><li>ambient composition : Dark fully visible image, but there's one single source of light</li><li>dun composition : Dark fully visible image with diffused light, no apparent light source</li><li>dark composition : Fully dark, no light source, close to pitch black image</li><li>contrast composition : Contains both dark and bright parts in the image, the dark and bright doesn't necessarily need to work together (ex: heaven &amp; hell, day &amp; night, or just a big shadow)</li><li>bright composition : Very bright image with strong highlights (close to white highlights)</li><li>vibrant composition : Image with high intensity (saturated) colors for majority of the image, independent of light source</li></ul><h2 id="style-tags:-5vxf3s6zo">style tags:</h2><pre><code>illustration style, western style, anime coloring, realistic, photorealistic, bold lines, 3d, 3d blender, 3d koikatsu, 3d mmd, 3d filmmaker</code></pre>These tags were introduced/utilized to absorb styles that are different from what we wanted in the base model. We also tagged specific styles that were present in our dataset to properly separate it from the main style (list on technical details). If the generation deviates from the base style, you can include tags like "3d" and "western style" in the negative prompts. Sometimes the base Pony's knowledge leaks out, but we plan to document and fix these issues as we identify more untrained tags.<ul><li>illustration style : anime style image with basic shading (little to no gradients used for shade)</li><li>western style : any western styled images that doesn't match the base style</li><li>anime coloring : images with anime coloring</li><li>realistic/photorealistic : this model is not intended for this but we used these tags for hyperreal illustrations or photo looking images. followed danbooru's definition</li><li>bold lines : Common with western style but also in some illustration. used for images with thick lines.</li><li>3d : Images tagged with just "3d" are 3d images that didn't fall under the categories below.<ul><li>3d blender : 3d images made using blender</li><li>3d mmd : 3d images made using mmd</li><li>3d koikatsu : 3d images made using koikatsu</li><li>3d filmmaker : 3d images using 3d filmmaker</li></ul></li></ul>Translated tags:Since we don't live in a perfect world where all tokens are properly learned without concept bleeding, we made adjustments to few tags to train them accurately. The reasoning and a list of translated tags can be found in the technical details article, but here are a few examples:<ul><li>"torii" -&gt; split into "red torii" and "stone torii"</li><li>"clothed &lt;gender&gt; nude &lt;gender&gt;" -&gt; split into "nude &lt;gender&gt;" and "clothed &lt;gender&gt;" to better absorb them as separate concepts</li></ul>Known problems:Some tags were left untouched or are not sufficiently trained by our current model, causing it to invoke the base Pony's knowledge (e.g., "bimbo" and other minor tags not in the dataset). We plan to add images with these concepts in future versions to overwrite the base knowledge. The pseudo-signature problem from the base Pony model is weaker in our model but still present. We know it can be resolved with a combination of brute force and clever strategies to prevent bleeding, so we expect to solve it in the next version.<h2 id="version-history-and-what-to-expect:-ndn42frap">Version history, and what to expect:</h2>version history:<ul><li>Eclipse XL v2: TBA</li><li>Eclipse XL 1.3: changed lr for Te2 to be wayyyyy lower, this retained some pony, but the model seems a bit undercooked, compared to V1 (published 6/29/2024)</li><li>Eclipse XL 1.1 and 1.2: testing new config values and testing way to prevent deterioration of concepts. (Edit: we stopped training half-way for these cause their quality was worse)</li><li>Eclipse XL v1 (published 5/31/2024): this includes phase 0 ~ phase 2 of our project, using finalized config</li></ul><ul><li>Beta version (unnamed): phase 0 and phase 1 dataset, testing configs</li></ul>Eclipse XL v2 will include our phase 3 dataset which will incorporate the following:<ul><li>Weapons (swords, gun, etc)</li><li>More fantasy races, furry &amp; non furry, robots (gundam and stuff)</li><li>More angles</li><li>We currently have a long list of concepts to add, and we will take feedback for things that don't work in the current version and we will add to the list based on priority.</li></ul>What not to expect:<ul><li>We want to make a good "base" model, so anything that's entirely closed within a small circle/fandom is probably not on the list.</li><li>We don't care about supporting some random character from a minor show. Just imagine training every single character listed on Mudae, that's ~50 imgs per char x 110,000 characters = ~5.5 million imgs. That's a task for lora creators or look towards pony v7.</li><li>Same for random movie reference or something, the required dataset size easily stacks and they're a better task for loras.</li></ul><h2 id="version-log:-8x2tgtvc0">Version log:</h2><h1 id="current-best-model:-eclipse-v1-j1jehu2ar">Current best model: Eclipse V1</h1>Current test version: V1.3, we're testing LR and scheduler and stuff<h3 id="acknowledgements:-j58vxge28">Acknowledgements:</h3>Authors: <a target="_blank" rel="ugc" href="https://civarchive.com/user/Wasabiya">Wasabiya</a>, <a target="_blank" rel="ugc" href="https://civarchive.com/user/Hecatonchireart">Hecatonchirea</a>Testers: Nebuchadnezzar, and other anonymous peopleWe thanks <a target="_blank" rel="ugc" href="https://civarchive.com/user/Anzhc">Anzhc </a>and <a target="_blank" rel="ugc" href="https://civarchive.com/user/Shalie">Shippy</a> for helping getting started. We thanks people at deepGHS for their python libraries and models, which helped a lot. And many thanks to all anonymous people involved in or helped shaping this project.<h2 id="licenses:-zhh5pp2cm">Licenses:</h2>This model is licensed under a modified Fair AI Public License 1.0-SD<a target="_blank" rel="ugc" href="https://freedevproject.org/faipl-1.0-sd/"> </a>(<a target="_blank" rel="ugc" href="https://freedevproject.org/faipl-1.0-sd/">https://freedevproject.org/faipl-1.0-sd/</a>) license. Since we used Pony v6 as the starting point, the following modification to the license hold true: You are not permitted to run inference of this model on websites or applications allowing any form of monetization (paid inference, faster tiers, etc.). This applies to any derivative models or model merges.This is a first draft of the description. We will update the details if we find any errors or need to add more clarification.last edited: 6/29/2024

Files with SHA256: dc6c410e2b59e0fae4b7d44f2395127678c62a5e750ae7edcfd4bc228b361108