Support my work by buying me a coffee! Tossing even just a couple dollars my way will help me fund more Qwen Image Edit LoRAs. Thank uuuu ~~ <33
This LoRA (designed for the 4-step Qwen Image Edit Lightning LoRA for fast inference) creates a black and white matte (layer mask) of the prominent subject of the image. Once generated, go into your favorite image editing software (eg. Photoshop, GIMP, etc) and apply the output matte as a layer mask to fully remove the background of the subject. This is robust matting, meaning that it works across a wide variety of scenarios such as matting multiple subjects in the image, matting entire comic panels, mattes complex hair, among additional scenarios. Works on art, real pictures, animals, etc.
AI matting will never be perfect however, so on occasion you will have to manually adjust the mask by painting in with white or black on the layer mask, or by using curves on the mask. There may also be color fringing as well after the mask is applied, as that is the nature of matting. You'd have to manually decontaminate the colors yourself.
In order to use this, use it at strength: 1, CFG 1 @ 4 steps with Qwen Image Edit Lightning LoRA, and just input the following :
Select the subject of the image. The subject should be fully white, while the background should be fully black.This was only trained on selecting the prominent subject, so it can't select other parts of the image that you specify.
Description
Initial release.
FAQ
Comments (19)
This is really cool. I do hope you do a v2 which lets you select which object you want, but I think this is an excellent proof of concept. It works well.
Yeah, this was just a test on whether or not this could be even possible for Qwen Edit. Turns out it is, and I may have accidentally made a select subject model that competes with Adobe Photoshop's select subject feature 😅. v2 will come sometime when I complete a new dataset for it (which will probably take a bit of time i assume) which will allow you to just place a white dot on the image of what you want to mask out (or maybe just use text to decide on what to mask out... hmm 🤔).
@lilylilith I see two ways you could test the next version. 1) make the selection text based. 2) make the selection visual based. For 2, you could give it 2 images, a clean one, and one where you do a red (or whatever named color) scribble in the object you want to select. Then it uses the scribble version. This could even possibly allow integration with Krita or something in the future, to get a clean selection, but that's like 10 steps ahead.
Number 2 could have many approaches though, including single image, since the result doesn't need to know all the interior visuals.
@Jellai By the way, I've decided on the interaction method for V2. It will be circle-selection based. Think of something like Circle to Search by Google, where you circle anything and it quickly looks up and gives details on the item you searched. It'll be like that but instead you circle what you want masked. Eg. You circle glasses, and it only masks the glasses. You circle the hair, and it only masks the hair; that sort of thing. It's much better than typing out what you want masked, as you may or may not know what the object you want masked is called. Also, there's a chance the text encoder might fail to understand as well. The dataset is half prepared, now I just have to actually prepare the target images then it'll be off to training!
@lilylilith That's great! Will it let you circle by hand, so you can demark the object if it's a weird shape? Or will it require a perfect oval/circle?
@Jellai When I mean "circle", I really mean "enclose the area of the object in an outline/marquee." It will not require a perfect circle, it will just require that you outline the general area of the object you want to mask. The outline can be weirdly shaped, too.
@lilylilith Perfect. That's what I was imagining when I described the hand drawn option. Very excited for this. Thanks for going through this effort.
@Jellai Just an update with progress: It's trained, but it's not great. I trained it on Qwen Image Edit Non-plus like I normally do, but this concept is just very weird for it to understand. It does somewhat the instructions I'd say two thirds of the time, but a lot of the time it leaves the residue of the circle in the mask (ie. the circle outline becomes part of the mask, which is annoying). I think I do know what to do to improve it:
1.) I need to double my dataset. My dataset had 50 pairs of images of objects/features outlined and the target masks. There are specific objects this fails at, so it needs more training data.
2.) It needs to be trained on Qwen Image Edit Plus. It definitely needs a "clean" image for it to understand the context, as it doesn't understand having an outline around the object. Unfortunately that means the UX will be hindered as you will need to provide both a "clean" original image and that same image except with an outline around the object you want masked. It would have been nice to just have to provide one image with the outline and get the mask as an output but alas, it doesn't look like it works the best.
3.) It needs way more steps, at least 5000 steps. I trained the other model at 3500 steps, but it looks like it needs more steps to converge fully.
@lilylilith Hmmm... I wonder what the difference is between your approach, and loras like "Put This Here", which basically do the same thing by surrounding the object with white, or even Image Fusion, which is a LOT more subtle, with the only thing identifying the object is that the lighting isn't exactly the same as the environment, or doesn't cast light/shadow in the environment:
https://www.reddit.com/r/comfyui/comments/1ohf8f4/qwenedit2509_image_fusion_lora/
That's far far more subtle than a circle. Here's a tutorial about how they made the datasets:
https://www.reddit.com/r/comfyui/comments/1ofw6so/explain_in_detail_the_idea_of_making_data_sets/
https://youtu.be/NEaDxgdMadY?si=VrFk82XHy_1Wcrf
There has to be some difference in approach that's causing problems with your lora results. It seems clear that Qwen can be trained to do far more difficult, yet similar things. I'd like to help in some way.
@Jellai Btw i'm stopping development of v2 in favor of waiting for Z-Image Edit (since I believe it will be MUCH better than qwen image, and won't take that long to release I think). If you want control of the object selected, the recently released Nano Banana Pro model by Google does insanely well. Just tell it to "create a layer mask of _____", where the blank is whatever you want masked. It's pretty crazy how well it does.
@lilylilith I think that's a smart move. I agree it could be better than Qwen Image Edit (even though a new one is coming out this week). In no small part due to the speed and depth of training, as we've seen in Turbo. Z-Image trains so well.
this one amazingly works closed colors and complex background all good, qwen edit is great then loras like this, oh man loving it, thanks for creating this lora, plz share more loras, thanks again
I will be making more LoRAs for Qwen, don't worry! Also thanks for the support :3
Чувак Лора хороша но сделай чтобы оно делало так. 1) Вырезает персонажа убирает задний фон 2) Затем по описанию добавляет фон и правит свет используя эту https://civitai.com/models/2076009/qwen-image-edit2509-relight
Следующим 3) Расширяет изображение используя https://civitai.com/models/2106308/uncropinpaintoutpaint-with-context-image-for-qwen-image-edit-2509-lora?modelVersionId=2382934
4) Затем последний выдает результат.
Я знаю что можно такое сделать, но не знаю как правильно собрать в нодах. Буду благодарен за такое Workflow. Да и другие тоже.
hi, sorry i don't see the usability here? there's ton of matting tools that does that, could you precise the improvements here or difference?
Podes compartir un flujo de trabajo?
@lilylilith thank you for making this public - may i ask you what is the original model (which version of qwen image edit - 2509 or original?) and lora used to run this model?
This LoRA only supports the original version of Qwen Image Edit (prior to 2509 & 2511)
@lilylilith thank you!