🎭 Emotion Puppeteer - For Krea2 and SDXL 🎭
Krea2 - How to use (见下文中文)
Krea2 can already make complex facial expressions. The purpose of this lora is to make the emotions more realistic.
If you use vanilla krea2, then a refusal reduction lora is required for this lora to work. All popular finetunes already have refusal reduction baked in.
Prompting v2
The gallery images have embedded prompts.
Start simple: describe the mouth & eyes
👁️ eyes wide, winking, squinting, half-lidded, scrunched, eyebrows raised / lowered / arched
👄 smirking, smiling, laughing, shouting, breathing, gasping, pouting, puckering, frowning, sneering, snarling, growling
Add more face descriptions as needed
combine two mouth or eye shapes, e.g. "shouting and frowning", "squinting with eyes closed"
Add micro-details as needed
add "wrinkled" or "creased" to the eyes, nose, chin, or forehead — for extreme anger/sorrow
prompt the left and right sides of the face differently (e.g. for sneering, winking, smirking)
prompt the upper and lower teeth visibility, flared nostrils, cheeks
the more words you add, the better the "acting"
Emotion words don't matter much for v2
basic "sad", "happy", "angry" still works, but not as strong as describing mouth and eyes
Troubleshooting
Problem: Emotions are weak
Solution: You must use a refusal reduction lora with the vanilla krea2 model
Solution: Increase image resolution. If the face doesn't fill the most of the image, 1MP is too small. 2MP is recommended.
Problem: Faces are blurry / plasticy
Solution: Increase image resolution (2 megapixel recommended if the face is small in the image)
Solution: Decrease the lora's strength (0.7 to 1.0 recommended)
Problem: Face aren't photo-realistic
Solution: Add any of the many realism loras
Question: Does this work with character loras?
Answer: It depends on the lora. Some loras have very inflexible faces. If this lora changes the face identity, turn down the strength.
Krea2 - 使用方法
Krea2 已经能够生成复杂的面部表情。这个 lora 的目的是让情绪更加真实。
如果你使用的是原版 krea2，那么这个 lora 要生效，必须搭配 refusal reduction lora。所有热门的微调模型都已经内置了拒绝减少（refusal reduction）功能。
提示词写法 v2
示例图库中的图片已内嵌提示词。
先从简单的开始：描述嘴巴和眼睛
👁️ 眼睛睁大、眨眼、眯眼、半闭眼、挤眼、眉毛上扬 / 下压 / 拱起
👄 微笑、咧嘴笑、大笑、大喊、喘息、倒吸气、撅嘴、抿嘴、皱眉、冷笑、龇牙、低吼
根据需要添加更多面部描述
组合两种嘴形或眼形，例如"大喊且皱眉"、"眯着眼睛闭眼"
根据需要添加微表情细节
在眼睛、鼻子、下巴或额头加上"wrinkled（起皱）"或"creased（有褶皱）"——用于表现极度的愤怒/悲伤
对脸部左右两侧分别描述（例如用于冷笑、眨眼、坏笑）
描述上下牙齿的露出程度、鼻翼张开、脸颊
添加的词汇越多，"表演"效果越好
对 v2 来说，情绪词并不太重要
基础的"悲伤"、"开心"、"愤怒"仍然有效，但效果不如描述嘴巴和眼睛来得强
故障排除
问题：情绪表现很弱
解决方法：使用原版 krea2 模型时必须搭配 refusal reduction lora
解决方法：提高图像分辨率。如果人脸没有占据画面的大部分，1MP 就太小了。推荐 2MP。
问题：人脸模糊 / 塑料感
解决方法：提高图像分辨率（如果人脸在画面中较小，推荐 2 兆像素）
解决方法：降低 lora 的强度（推荐 0.7 到 1.0）
问题：人脸不够照片级真实
解决方法：添加任意一款写实类（realism）lora
问题：这个可以和角色 lora 一起使用吗？
回答：取决于具体的 lora。有些 lora 的面部非常不灵活。如果这个 lora 改变了面部特征，请降低其强度。
SDXL lora info
How to use
Use this visual cheats sheet showing possible 96 eye-mouth combinations
👁️ grieving, glaring, gazing, seething, beaming, eyes wide, winking, squinting
👄 frowning, pouting, smiling, laughing, singing, screaming, sneering, snarling, grimacing, breathing, gasping, puckering, resting, relaxing
Add one more 👁️ words and/or 👄 mouth words to your prompt
For a stronger effect, increase the weight of the 👁️ / 👄 words, and/or increase the lora weight up to 1.2
Suggestions
Best for realism. Most training images were photos
For character consistency, use a combination of controlnets
Keep your negative prompt short
Other optional prompts
These words were all used in training, but their effect is weaker:
gnashing, cackling, flirting, glowering, leering, kissing, shrieking, shouting, smooching, sniffling, threatening, tisking, taunting, vomiting, wincing, yelling, accusatory, agony, amused, angry, anguish, annoyed, appalled, approving, aroused, ashamed, astonished, awestruck, bemused, bewildered, bitter, blame, certain, cheerful, cocky, concentrating, concerned, confident, confusion, contempt, contemptuous, content, crazed, crestfallen, crushed, curious, dazed, defiant, delighted, depressed, despairing, determined, disappointed, disapproving, disbelief, disgusted, dismayed, distrust, disturbed, dominant, ecstatic, embarrassed, enraged, envious, euphoric, evil grin, exasperated, excited, flirtatious, frightened, frustrated, furious, gentle, gleeful, happily, happy, hate, heartache, heartbroken, helpless, horrified, hurt, hurting, impatient, impressed, in love, incredulous, insane, insolent, intrigued, jubilant, judgemental, loathing, lustful, merry, misery, mistrust, mocking, nauseated, nonplussed, outraged, overjoyed, passionate, perplexed, petulant, pissed off, pitying, playful, puzzled, regretful, repulsed, revolted, rude, sad, scared, scorn, seductive, seizure, shaken, shocked, sickened, silenced, skeptical, smarmy, sour, startled, stunned, submissive, sultry, surprised, suspicious, terrified, thinking, ticked off, uncertain, unconvinced, unsettled, vexed, violent, wonderDescription
Better captions, more steps
FAQ
Comments (3)
佬，屌！还需要更多的表情，最好把红黑小情绪给融了 ❤️
Wouldn't it be simpler to aggregate all those micro-movements and train the model directly on the emotions that humans innately recognize and identify? In reality, hardly anyone can construct a prompt for "anger" or "joy" based solely on the positioning of eyebrows, mouth corners, and the nose without having visual references right in front of them. While this knowledge is hardwired into the subconscious, most people aren't trained to break it down into specific details. Micro-movements could serve as an enhancing factor, with basic emotions acting as the foundation of the prompt. After all, if you drill down far enough into micro-details, you eventually arrive at the level of individual pixel placement—and for that, you don't really need neural networks.
I agree with you, but it's hard. I welcome suggestions.
As of v2, this lora is hard to use, but it satisfies my first goal: it allows you to generate a wider variety subtle and more natural facial expressions. Even if you use simple prompts like "anger" or "squint", in my opinion the output looks better than vanilla krea2 and better than any other emotion lora available.
My second goal is to make this lora easy to use. Here's why that's hard: The current dataset has 80 distinct emotions, ~4 images per emotion, grouped into 9 broader emotion categories (e.g. "anger"). 11 emotions fall into the "anger" category, but they actually bridge other broad categories, e.g. "disgusted anger" and "happy anger".
If I simply caption those 11 types of anger (~40 images) as "anger", the result is generic looking anger that's no better than what vanilla Krea2. And that's probably how Krea2 and every other image model was captioned, and why its emotions look generic. That captioning collapses "disgusted anger", "happy anger", and several other kinds of anger together into one generic stock photo actor anger.
I could caption all 80 emotions separately, but that would be hard to use too. Who could remember that producing "disgusted anger" needs the keyword "disdain"? I think a good approach will be to caption every emotion as a combination of 3 categories, e.g. "mostly angry + a bit happy + a bit disgusted". I think that would work. Either way, I'm going to need a 4x larger dataset, or ~900 more images. That'll be very hard to produce/collect and caption.
So the current compromise is to caption small features of the face, since several of the 80 emotions share each feature. For example, some degree of "squinting" is present in ~20 of the emotions, and ~80 training images. That compromise makes this lora hard to prompt, but I think easier than drawing with pixels. Honestly, I think mine is the best emotion lora currently available, but I would be very happy if someone else creates a better one!