Zonkey V7 - Download at your own risk edition.
Trained another slider on base Pony with 1.5MP buckets instead of 1.0MP buckets and applied it to Zonkey 6, . Also applied NTC's Not Simple Background LoRA.
Model likes low CFG.
I found that using score tags in an initial prompt, and performing a restart of most steps using a prompt without them, can produce some nice results, in exchange for slightly less sharpness of detail. Examples in the images with embedded ComfyUI workflows.
Zonkey V6.2 - Hyper 8 step edition
Incorporated the ByteDance Hyper 8 Step LoRA, and NTC's Not Simple Background LoRA, for better backgrounds. I recommend Euler a for first pass, with a CFG of 1.25-1.5, and DPM2++ SDE, or similar for upscaling. Upscaling is recommended. For 2 character scenes, 12 steps can help sort out character details but 8 steps is generally fine for 1 character/pov scenes.
Zonkey V6 - Save the day edition - Now with 50% less sameface
Decided to be less lazy, and do some training with v6. I injected 25% of Pony back into Zonkey v5. I inpainted the faces on 400 photos, 200 close, and 200 far, with this model and used the resulting pairs to make a 2 slider loras, to make things look less pony-ish. The loras were then merged in at different layers.
Zonkey V5 - Baked from scratch edition
Started with all the ingredients of Zonkey V3 and DARE merged them all in random order at 6% each for 50 iterations. Added 12.5% halcyonSDXL_v17 and 5% bemypony_real. Back to original Pony CLIP, so it should train LoRAs better than v4. Brought the brightness back up, for a more neutral style, and reduced the noise from 4.2.
I recommend either not using score tags with this version, running them for an early fraction of steps, or placing them near the end of the prompt. They can help with composition, but they nudge the image towards anime style. 1girl has a similar effect, so leave it out if you don't need it. The hashed token negative I used with previous versions tends to break things now, so I've stopped using it as well.
Zonkey V4.2 - Double Check Your Work edition
The extra noisiness of 4.0 was due to an incorrect setting on one of my CLIP DARE merge nodes. Fixed that, and snuck in a little extra oneFORALLPonyFantasy_v20DPO into the CLIP. 4.2 improves clarity, and a standard number of steps can be used again. All example images(except for the first one) use the same generation data as the 4.0 images, except using 30 steps for both first pass and Hires fix, instead of 40-100. They aren't cherry picked, so there are a few glitches I normally would have left out, but I wanted to show a direct comparison.
Zonkey V4 - Beat to the Punch edition.
Was gonna make a model called Godiva, but the name got scooped. Oh well, this model kicks ass, whatever it's called. Very little character accuracy was sacrificed for the photorealism. The CLIP is now modified. I replaced ~50% ofthe original Pony CLIP with oneFORALLPonyFantasy_v20DPO's CLIP and another 25% with proteusRundiffusionDPO_truereversecubich. Added ~40% proteusRundiffusionDPO_truereversecubich, ~10% datassRev3Pony_rev3, and ~5% damnPonyxlRealistic_damnV20EXTREME to the UNET, V Gradient merged it back with Zonkey V3 and put ~25% enjoyXLSuperRealistic_v30ModifiedVersion in the out layer.
This model is stuffed with creativity. It can be a little noisy, use Euler a, for the Hi-Res fix, instead of a DPM++ sampler if it's too much, I think I found a good balance between noise and creativity. It likes lots of steps, I'd suggest at least 30-40, and Hi-res fix, but overall, the anatomy is more reliable than any previous version, so you shouldn't have as many wasted generations if you give it those extra steps.
Enjoy and get wierd with it. I like seeing all the things that get made with it, whatever it is, so don't be shy about posting.
Zonkey V3 - Mad Surgery edition.
Zonkey V2 got a DARE injection of a CosXL merge of Copax Timeless, Rundiffusion Proteus, Art Universe, Realistic Stock Photo, and People Photography, add difference, paired with DARE removal of Jibmix, AnimeBoys, ToonSphere3D and Animagine. Then it was all DARE merged back into original PonyDiffusion V6 XL.
V3 is more versatile than previous versions, capable of achieving both higher photorealism, and more toon-like styles, but it has to be prompted for. Using the terms "real", or "real life" near the beginning of the prompt, without score tags, can help achieve high levels of photorealism, with better faces, especially when weighted fairly heavily. Using score tags can improve overall aesthetics, but will reduce photorealism, and give faces more anime/cartoon-like shape.
Along with improvements to contrast and color range, Zonkey V3 includes the Blessed VAE to amplify the effect.
Zonkey V2 - Big Hairy Juggs edition.
New Masked DARE injection of Juggernaught X, SDXXXL, Bordello and Ratatoskr. U-Net renormalization relative to PonyDiffusionV6 XL.
Improved photorealism
Better lighting
Sharper detail
Higher background variety and detail
More vivid colors
Fewer artifacts
Better anatomy
Higher character accuracy
Furrier furries
Zonkey was created with the goal of bringing as much photorealism as possible to a Pony model, while attempting to retain its flexibility and prompting power. Like other Pony realism models, it can have problems with eyes when faces are relatively small, but Hires fix often improves them.
All posted images are DPM++ SDE Exponential, 30 steps, CFG 3.5-5.0, Hires Fix Latent(bicubic antialiased), 1.5 or 2x. It can be easier to get better poses with Euler a and DPM++ 2S a, but the detail isn't as high. No ADetailer was used, but it may be helpful in some cases. See posted images for suggested prompting style.
The following checkpoints were used:
Animagine XL v3.1
Art Universe SDXL v2.0
Bordello v1.6
CinematicRedmond v1.0
ChacolRealPonyMixXL (Asian Version) v3.0a
Copax TimeLessXL v11
FULLY_REAL_XL v9.0
Juggernaut XL v8
Pony Diffusion V6 XL
Ratatoskr v3.8
Realistic Freedom Wonderland
RealVisXL v4.0
RunBull_XL v0.4
RsmPornXL v0.81
SDXXXL v3.0
Virile XL v1.0
yudas_woman v3
ZavyChromaXL v6.0
Along with the following Loras:
BoringReality_faces v4
Porn Productivity(multi-concepts) PP-21 v1
RMSDXL Photo XL v1.0
SDXL Offset Example Lora v1.0
Styles for Pony Diffusion V6 XL Photo v2
The Handsomizer v1.0(Pony Diffusion V6)
Description
Zonkey V4.2 - Double Check Your Work edition
The extra noisiness of 4.0 was due to an incorrect setting on one of my CLIP DARE merge nodes. Fixed that, and snuck in a little extra oneFORALLPonyFantasy_v20DPO into the CLIP. 4.2 improves clarity, and a standard number of steps can be used again. All example images(except for the first one) use the same generation data as the 4.0 images, except using 30 steps for both first pass and Hires fix, instead of 40-100. They aren't cherry picked, so there are a few glitches I normally would have left out, but I wanted to show a direct comparison.
FAQ
Comments (82)
What's the weird negative prompt for?
What I remember is that someone figured out you could hash certain tokens with those 3 letter strings, so people put them in the negative to avoid them sneaking into images. That's probably oversimplified but hypothetically you could enter one of those strings in the positive and get like "tracer doing yoga" injected into the prompt encoding.
i think v4 is the best one yet, i actually like the grain, it makes pictures more realistic to me.
any tips to remove the film grain and hard shadows? even when placing in negatives everything ends up looking like its in a half lit room
Use the standard SDXL vae, instead of the custom one I baked in, and use the SDXL noise offset Lora at negative weights https://civitai.com/models/137511/sdxl-offset-example-lora
4.2 has less stability, but more detail.
I trained 4 face LoRAs using Zonkey 3.0 as my base and they came out perfect. I had no problem generating the same level of realism when using the LoRA with the Checkpoint. However, when I trained a face LoRA using 4.2 as a base my LoRA came out looking extremely cartoonish. Any idea why this might have happened?
Zonkey 1-3's CLIP is unmodified from base Pony. Zonkey 4+'s CLIP is heavily modified(which is why I initially intended to release it as a separate model), mostly with https://civitai.com/models/380964/proteus-rundiffusion-dpo-direct-preference-optimization
Proteus was trained with captions made from short, comma separated tags, like Pony, but they aren't Danbooru tags. While 4.2 retains pretty good compatibility with Loras trained on base Pony, I'm not sure of best practices to train for it. There don't appear to be any Loras made for Proteus.
@bot_i_celli ah ok thank you for the explanation. Love your model! Great work!
Do all generated images look this dark in V4 and V4.2?
I may have overcompensated for the blown out look of other Pony realism models. Part of it is the custom VAE. You can use the standard sdxl VAE to increase brightness and reduce contrast. Beyond that, the EnjoyXL that I put in the output layer reduced the brightness. You can counteract this somewhat by using the SDXL noise offset lora at negative weights https://civitai.com/models/137511/sdxl-offset-example-lora. There are some other brightness/contrast Loras, that I haven't tested, that likely work too.
@bot_i_celli standard sdxl VAE indeed looks much better, thanks
I've not been able to get 4.2 to work yet, characters fail to connect most of time and not figured out why. thus I'm still using 4.0, but omg it's good! Will update if I figure out why.
Solved the issues which were entirely self-created, just 4.2 was less forgiving of my stupidity :)
Trainers beware. I trained a character on this version of the model and all of the images come out looking like GigaChad. I don't have a single usable epoch. I'll try to see if prompting can fix it, but the bias towards massive cleft chins in this model is huge. As much as I like what the model can do, it will make your AI influencer portraits come out looking like they belong in JoJo's Bizarre Adventure.
Wait, having GigaChad meme face is a bad thing?
@Suppressor On a case-by-case basis, I suppose it can be great. lol And to be fair, not quite all of them looked like GigaChad... There were some Handsome Squidwards, as well.
@AIngelsandDaemons i hate the SD tendency to give everyone cleft chins, no idea where that has come from but a lot of pony merges end up pulling that stuff in and i avoid it like the plague because its impossible to remove with prompting, honestly surprised there arent more loras to deal with it but I think other people dont notice it as much as i do
@eye_am_bored @AIngelsandDaemons I forgot to add it back into v4 and v4.2, so it got diluted from v3, but the Handsomizer Lora, at negative weights, can help avoid the gigachad cleft chin look.
@AIngelsandDaemons Handsome Squidwards, lol!
@AIngelsandDaemons It doesn't make sense to train on variants of Pony. Train on base Pony and your Lora will work perfectly with any variant of it.
This is just base itself with a Lora where all the training data was actual photographs of an assortment of similar-looking real people, for example, and you can look at the gallery for the Lora to see examples of it working with different Pony variants.
@diffusionfanatic1173 Thanks for the advice! That was going to be my next attempt, but it's nice to hear this from someone with more experience than me. I'm still pretty new to the training game.
@AIngelsandDaemons No problem
@bot_i_celli thanks for the advice, this wasnt meant to be an attack on this model specifically, it seems to be weird trend with all versions of SD from 1.5 to XL that is more or less pronounced in different merges
I don't think this got fixed... if anything V4.2 doubled down on the noise and is way more noisy than V4.0!
I will say I don't understand in what way the model author thinks this image for example looks in any way good, it's a mess:
There is something wrong with red colored blotches appearing sometimes, which only go away after you put the score tags in the negative prompt. It's very weird.
I noticed red patches in v4.0, I was always able to get rid of them by adding more steps.
@bot_i_celli I think I prefer v40
No Pony model comes even close to this one in terms of having BOTH great image quality and keeping the Pony prompt recognition intact. This model keeps both the nsfw stuff and even the characters. This model is criminally UNDERRATED. How come a model like everclear is so much more popular than this one.....? The creator of this model you are a wizard.
Thanks! Zovya is the #1 base model creator, revAnimated is a fairly legendary model, so I'm not surprised that reputation gets his other model downloads. Beyond that, I think I tried too hard for branding, and put an anthro zebra on the preview image of the first version. I should of just had a portrait of a pretty blonde. Also, I didn't include Realism in the name until V4, so some people had trouble finding it. I've seen a few people reference it on reddit and 4chan in the last few days, so I think more people are learning about it.
Your tags definitely opened this model up! The detail LoRA's you use with them are also amazing.
Seems like more realistic than Everclear but needs some improvments for sure.
It is way better than Everclear. By orders of magnitude.
So been testing both V3 and V4.2. Both looking really good, 4.2 looking more realistic.
Both follow the prompt extremely well (but give fairly varying results).
One of my favorite models atm, doing some more testing and using it to finetune a person next.
Backgrounds also look significantly more detailed than most other realistic pony models.
Great work so far!
Other people have had problems fine tuning on v4.2. Might be better to finetune on v3, they should be fairly compatible.
@bot_i_celli Yea, I've read it but I want to test it anyway tbh :p Testing an early epoch, likeness isn't fully there yet but it doesn't look wonky either so far.
Update: Still not finished but last epoch looks pretty damn good tbh. No weird cartoony faces as someone reported.
Found it and Thank You.
One of the best models for me...!
I like the sketchy challence...
trying to find a way through it....!
Very creative model, for sure.
Images and captions must have been something special...!
It's so f*cking good, I only found glitches with some characters, everything else goes so hard, is extremely detailed and has almost no issues (except eyes). Would be awesome to see a 2.5d merge and an anime version of this model.
Try using this prompt:
POSITIVE:
score_9,score_8_up,score_7_up,score_6_up, wide view,
[YOUR PROMPT]
"large eyes", "even eyes", "aligned eyes", "correct proportions", "perfect anatomy", "natural pose", "proper angles", "accurate perspective", "facial symmetry", "straight limbs", "regular features", "well-proportioned body", "balanced perspective", "proportionate features", "natural anatomy", "straight limbs", "straight body parts", "aligned features", "even eyes", "normal proportions", "accurate perspective", "anatomical accuracy", "balanced proportions", "accurate perspective", "proper alignment", "well-defined features", "aligned perspective", "correct anatomy", "balanced features", "natural proportions",
"natural poses", "proper proportions", "matched eyes", "high resolution", "clear image", "straight limbs", "natural stretches", "smooth features", "aligned eyes", "high clarity", "sharp details", "clean edges", "accurate anatomy", "balanced proportions", "correct perspective", "smooth texture", "consistent colors", "artifact-free", "clean pixels"
NEGATIVE:
disfigured, ugly, contortion, stretched, duplication, artefacts, blur, incoherent, "contortions", "stretches", "mismatched eyes", "pixelation", "image artifacts", "distorted limbs", "unnatural stretches", "warped features", "misaligned eyes", "low resolution", "blurry details", "compression artifacts", "jagged edges", "unnatural anatomy", "skewed proportions", "distorted perspective", "grainy texture", "color banding", "artifact noise", "broken pixels"
,"small eyes", "uneven eyes", "misaligned eyes", "incorrect proportions", "distorted anatomy", "awkward pose", "unnatural angles", "perspective errors", "facial asymmetry", "warped limbs", "irregular features", "misproportioned body", "skewed perspective", "disproportionate features", "exaggerated anatomy", "twisted limbs", "bent body parts", "skewed alignment", "crooked eyes", "abnormal proportions", "distorted perspective", "anatomical errors", "uneven proportions", "warped perspective", "improper alignment", "disfigured features", "misaligned perspective", "incorrect anatomy", "skewed features", "unnatural proportions"
Before I go for it, what does this achieve?
@Suppressor The quality of my pictures. It seems to fix eyes and hands and duplicates for me.
@virtualfix6885 Oh cool, the eyes and duplicates have plagued me on a few generations. I would mention that euler fixes any problems you might have with LoRA compatibility as well.
Thanks it worked for me too!
@Suppressor thanks for the tip I will try this too
@Suppressor I'd say it would dilute the hell out of any real prompt you have. Less is always more.
@dc802 I thought that too, but it doesn't seem to dilute the prompt for me. I always get what I ask for, maybe give me a complicated prompt to test.
seems unnecessarily overcomplicated. You can also group tags. So if you have any tags that use the same word, scrap the duplicates and group them. In this situation I'd make it, mismatched-misaligned-small-uneven-misaligned-irregular-crooked-eyes. But even better, I would leave the eyes part out entirely because It increases the odds that the character's eyes won't be visible at all instead they'll be seen from behind or far away. Also, mismateched-eyes could be heterochromia. Grainy-texture, filmgrain, jagged-edges, antialiasing-pixelart. And I suggest to include, trauma-handicap in each prompt. :P Hope it helps a bit. ^^
@c666d https://civitai.com/images/14745285?postId=3273336 It seems to work fine for me.
What's all this guff? (gpo, aca, aer, api, fla, gcx, hll, hnj, gpc, fii, fey, fbv, evg, iew, ifl, igh, iwj, iwp, ixb, ixe, ixz, jaf, jbm, jfb, jsf, jyk, kmz, ksh, kxg, kzg, lbv, zac, yle, zmj, szw, uiw, vfe, par, pdl, qdl, mbo, mtd, gor, bhz, dit, frw, fnaf, bmo, zbi:0.5)
style tokens
@xJollyboyx and what purpose do they achieve or is it really just more word salad like 50% of what makes up prompts on Civitai?
@dc802 guys on 4chan made this sheet:
https://lite.framacalc.org/4ttgzvd0rx-a6jf
it has pics and short descriptions along the tokens, bit of a mess but helpful
@xJollyboyx One look at that and I'm just going to ignore it. Sampled a few with and without and there isn't any consistent gain. Prompting placebo
@dc802 false, some are quite strong
@xJollyboyx out of the 100s of options. Yeah nah.
@dc802 oh btw, you should use them with a base model like v6 or autism
one of the best if not the best currently at preserving the pony wild NSFW stuff while getting quite close to photorealism, well done
Great model overall, in my experience it's the best realistic Pony model, skin details and lighting are exceptional. The only problem I've found is that emotions and facial expressions seem to be very subdued and weak. Especially extreme sexual emotions like ahegao, orgasm, moaning, tongue out, etc. They kinda there but don't look very good. Tears also look very unnatural. And I do like crying girls...
Super good,Very close to realism,still kinda have some problems with eyes tho
You can easily fix that with higher steps and or strong prompting. I fought with that as well.
@Suppressor oh,I thought I only had to use 30, thanks tho I will try
@kd231002573 Yeah for sure, I've been pushing 44 and up on my generations and getting very few bad results.
@Suppressor Interesting,I will do it then
Use ADetailer
V4 was good, but V4.2 is amazing! It's just as you described! It understands the prompt better, improved the backgrounds, textures as well as the details! <3
Using "real life" in your prompt affects poses. But if you use [(real life:2.5):3] in your prompt, you delay the "real life" keyword from taking effect for three steps, and you get Pony-model poses instead. Confirmed working on Zonkey V3. If you use more steps, pick a higher step count than 3 (around 15% of the total number of steps).
you can then use [(real life:2.5):0.15] to account for that % delay
You think you could merge LEOSAM V7 into this model? And maybe make another merge that makes a more painterly style? Your model is literally the best realistic Pony version.
Yes please
Make this available for on site generation please
I seem to have a problem when attempting to use Control net. Is this an issue with the checkpoint, pony in general or am I possibly doing something wrong?
from what i gather Controlnet isn't friendly with Pony in general, use scoring and style loras
Thank you. I'm not sure how style LoRA would benefit me considering I only use Zonkey and generate photorealistic images. I was wondering if anyone knows if it's possibly to train a LoRA on pony or Zonkey to try to get better background settings. I've only done clothing, face and hairstyle LoRA
@arielgoodblow287 I'm actually trying to do work towards that specific goal. https://civitai.com/models/503874/pony-realism-backgrounds
May help with your backgrounds, definitely helps with mine.
@Suppressor ooo thank you. Did you just train with scenery images in your datasets? do you caption everything? Do you use a trigger word?
@arielgoodblow287, I used scenery, texture samples, multiple shot angles, close ups, and individual elements (like leaves or snowflakes etc) that compose the scenes to try and maximize realism. There is a general trigger word for each scene, plus modifiers based on captioning listed on each model. The captioning was half human, half AI generated, and may get tweaked as I collect feedback.
⠀⠀Let's compare some PONY models 🦄
⠀⠀⠀⠀🌍 See here my detailed test 🌎