AI Gimp Plugins

[nchen]

Hello,

I created some plugins that utilize ComfyUI to add AI features to GIMP. This includes object erasing, image generation, and image-to-image generation. I have some videos on my Github: https://github.com/nchenevey1/gimp-comfy-tools

Features are limited by imagination. I plan to add upscaling in the future, which can reliably create good AI photos above 8000x8000 or improve low res images. I'm an impoverished student though, so allow me patience if you have a feature request.

Personally I don't think AI art is unethical, but this can assuage those qualms with open source and local generation. The downside is that this does require access to VRAM locally or through Websocket, as well as storage space for big models (usually 2-10 gigabytes each). It works well for me with an RTX 3060 laptop GPU.

If anyone likes this and would like help to set it up, let me know.

I hope you all find this interesting!

[inkdolls]

(10-10-2024, 05:44 PM)nchen Wrote: If anyone likes this and would like help to set it up, let me know.

Well, I'm not getting far at all. When starting ComfyUI, I get this error:

Code:

Cannot import /home/xxxx/src/ComfyUI/custom_nodes/comfyui-gimp-nodes-main module for custom nodes: module 'comfyui-gimp-nodes-main.nodes' has no attribute 'SendImageDimsWebSocketGIMP'

There's no class "SendImageDimsWebSocketGIMP" in nodes.py, so I guess there's an error there.

Thanks.

[nchen]

(10-25-2024, 11:39 PM)inkdolls Wrote:

(10-10-2024, 05:44 PM)nchen Wrote: If anyone likes this and would like help to set it up, let me know.

Well, I'm not getting far at all. When starting ComfyUI, I get this error:

Code:

Cannot import /home/xxxx/src/ComfyUI/custom_nodes/comfyui-gimp-nodes-main module for custom nodes: module 'comfyui-gimp-nodes-main.nodes' has no attribute 'SendImageDimsWebSocketGIMP'

There's no class "SendImageDimsWebSocketGIMP" in nodes.py, so I guess there's an error there.

Thanks.

Hey, so sorry about that. I messed up my last push and didn't upload the correct update. The correct nodes.py should now be available on my github and functioning. If you have any other issues or requests, let me know! I'll do my best to reply soon.

[inkdolls]

Thanks for the quick reply, and for the fix!

I was able to render an image from GImp, but then I couldn't load it back. I have problems with other comfyUI custom nodes, so I probably have to try a lot of things before I get back to you with a more concise issue, if I can't solve it on my own.

[nchen]

(10-28-2024, 03:16 AM)inkdolls Wrote: Thanks for the quick reply, and for the fix!

I was able to render an image from GImp, but then I couldn't load it back. I have problems with other comfyUI custom nodes, so I probably have to try a lot of things before I get back to you with a more concise issue, if I can't solve it on my own.

I'd be happy to help you troubleshoot. The image gets sent back using one of the "Send Image (GIMP)" nodes that is included. I recommend using the one with "dims" in the title because it helps with stability.

If you're using that node and it's still not coming through, check the Comfyui terminal and see if there is an error there. I tried to catch errors within the plugins but sometimes things fall through, whoops

[inkdolls]

Hey, thanks for your help.

I did some testing. My issues were I was using an old version of ComfyUI (your nodes appear now) and I hadn't even realized you're providing workflow files.

Unfortunately, even though I downloaded the rgthree nodes from https://github.com/rgthree/rgthree-comfy , I get errors because the "Lora Loader Stack" is missing.

It has been deprecated for a new node, the page says as much:
   


I guess that's an issue out of your control and you probably weren't aware of it. Sorry for the bad news.

[nchen]

(10-30-2024, 12:51 AM)inkdolls Wrote: Hey, thanks for your help.

I did some testing. My issues were I was using an old version of ComfyUI (your nodes appear now) and I hadn't even realized you're providing workflow files.

Unfortunately, even though I downloaded the rgthree nodes from https://github.com/rgthree/rgthree-comfy , I get errors because the "Lora Loader Stack" is missing.

It has been deprecated for a new node, the page says as much:



I guess that's an issue out of your control and you probably weren't aware of it. Sorry for the bad news.

Ah thank you for letting me know! This is a pretty simple fix so I should be able to fix it soon.

In the meantime, you could make a new workflow with a different Lora loader if you're able to edit the plugin python file. 

There is a function near the top for "workflow" that loads the lora node by its name. I think the code would otherwise work if the name was switched to a new Lora loader. 

I'm away from my computer for a while but will try to fix when I get a chance!

[inkdolls]

Ok, thanks for your reply, I guess that works, but I already had managed to make it work  before reading your post.

I found the last version of rgthree that still has the old node. Once I copied it, I finally got an image to appear in Gimp!

I've had a few hiccups though:

The checkpoint in the workflow overrides whatever I place in the dialog window in GImp. In fact, I initially got an error of 'Value not in list: ckpt_name: '1.5_dreamshaper_8.safetensors', since I don't have that model in my computer, it only worked when I manually edited the json file.

The Gimp plugin throws an error when reading json files that ComfyUI saves. I couldn't just save a modified workflow of yours, I had to manually edit the file.

Thanks a lot for this! This is going to be very useful. Sometimes I do a lot of back and forth between Gimp and Stable Diffusion.

[nchen]

(10-30-2024, 05:27 AM)inkdolls Wrote: Ok, thanks for your reply, I guess that works, but I already had managed to make it work  before reading your post.

I found the last version of rgthree that still has the old node. Once I copied it, I finally got an image to appear in Gimp!

I've had a few hiccups though:

The checkpoint in the workflow overrides whatever I place in the dialog window in GImp. In fact, I initially got an error of 'Value not in list: ckpt_name: '1.5_dreamshaper_8.safetensors', since I don't have that model in my computer, it only worked when I manually edited the json file.

The Gimp plugin throws an error when reading json files that ComfyUI saves. I couldn't just save a modified workflow of yours, I had to manually edit the file.

Thanks a lot for this! This is going to be very useful. Sometimes I do a lot of back and forth between Gimp and Stable Diffusion.

In regards to the checkpoint not working, are you supplying just the checkpoint name or the entire path? When selecting the file in the GIMP menu, it should provide the entire path. The way I currently have it, the code checks the provided path and selects just the file name at the end. Ultimately, just the name is necessary so this is an oversight on my part.
Another possibility: The code looks for the CheckPointLoaderSimple node (default checkpoint loader). Using another one might not work.

For the ComfyUI json files, you'll have to save as an API file. It should be one of the "save" options on the ComfyUI menu.

Also, I updated the files so it should work with the up-to-date rgthree power lora loader. I'll try to make the node options more flexible in the future.

You're welcome! Hopefully we can get it up and working smoothly for you. I appreciate you helping me with bug fixing

[inkdolls]

Okay, I know why the checkpoint was not being replaced: on line 26 of gp-generate-comfy-image.py, it says "CheckPointLoaderSimple" with a capital P in "Point", but the node is named "CheckpointLoaderSimple". Once I edited it, it runs fine. Understanding that part of the plugin was interesting.

I'll try to give you some more useful input about the workflow files. I realize now that modifying those is not very important for the basic stuff.

So for now I've just tested the "Generate-image" tool. Testing the others will probably take me some time. I'll get back to you if I run into other issues.