Plugin to integrate Gimp with Meta Segment Anything

[khemadeva]

(10-10-2023, 03:43 PM)denzjos Wrote: Installed CUDA (coffee time.............). Got this error after trying to install PyTorch on the windows command line : Can't find pip3. Please control if you used the right name and try again.

Please check this for pip3 on windows (I got this by doing a search):
https://stackoverflow.com/questions/4150...on-windows

I guess you will get most of the issues resolved by doing a search on net. You can still let me know if you are stuck at any place and I will try to help as far as I can. But since I don't have Windows, I won't be able to check on the windows specific issues.

BTW you need to install CUDA only if you have GPU.

[denzjos]

(10-12-2023, 01:19 PM)khemadeva Wrote:

(10-10-2023, 03:43 PM)denzjos Wrote: Installed CUDA (coffee time.............). Got this error after trying to install PyTorch on the windows command line : Can't find pip3. Please control if you used the right name and try again.

Please check this for pip3 on windows (I got this by doing a search):
https://stackoverflow.com/questions/4150...on-windows

I guess you will get most of the issues resolved by doing a search on net. You can still let me know if you are stuck at any place and I will try to help as far as I can. But since I don't have Windows, I won't be able to check on the windows specific issues.

BTW you need to install CUDA only if you have GPU.

khemadeva, thanks for the aswer. I won't spend to much time on installing this plugin. Maybe I try it another time again because it's timeconsuming.

[LeafAtWood]

Thanks khemadeva for the plugin, though I am having trouble registering it with my GIMP 2.10 on macOS 10.15.7. I have run your install validation script (/path/to/python3/python ./seganybridge.py vit_h /path/to/checkpoint/model/sam_vit_h_4b8939.pth) and got a Success. I have put the seganybridge.py and seganyplugin.py in the /Users/me/Library/Application Support/GIMP/2.10/plug-ins repository. I also changed the mode so that these files are executable (chmod 777). Made sure I restarted GIMP. I also made sure SAM was properly installled by running the script amg.py from the SAM github and generated the segmentation masks. So seems my only trouble is registering your plugin with GIMP. Do you have any idea why it is not registering with GIMP and would you know of any way to check/debug the registration process ?

I have relayed the news on my Deep_in_Depth twitter channel : https://twitter.com/Deep_In_Depth/status...9651147837

[LeafAtWood]

(11-07-2023, 03:49 AM)LeafAtWood Wrote: Thanks khemadeva for the plugin, though I am having trouble registering it with my GIMP 2.10 on macOS 10.15.7. I have run your install validation script (/path/to/python3/python ./seganybridge.py vit_h /path/to/checkpoint/model/sam_vit_h_4b8939.pth) and got a Success. I have put the seganybridge.py and seganyplugin.py in the /Users/me/Library/Application Support/GIMP/2.10/plug-ins repository. I also changed the mode so that these files are executable (chmod 777). Made sure I restarted GIMP. I also made sure SAM was properly installled by running the script amg.py from the SAM github and generated the segmentation masks. So seems my only trouble is registering your plugin with GIMP. Do you have any idea why it is not registering with GIMP and would you know of any way to check/debug the registration process ?

I have relayed the news on my Deep_in_Depth twitter channel : https://twitter.com/Deep_In_Depth/status...9651147837

Fixed the "registration issue" by just adding the following line to the top of each file seganybridge.py and seganyplugin.py
#!/usr/bin/env python

There also seem to be a problem with the default plug-ins path on macOS: /Users/me/Library/Application Support/GIMP/2.10/plug-ins   due to the space in the path 

When the seganyplugin.py calls the seganybridge.py it looks like this:
/Library/Frameworks/Python.framework/Versions/3.9/bin/python3 /Users/efe/Library/Application Support/GIMP/2.10/plug-ins/seganybridge.py vit_h /Users/me/DeepInDepth/SAM/sam_vit_h_4b8939.pth /var/folders/38/hws16c9j0qbfm9_r3t900slw0000gn/T/__seg__7Szrd4.png Auto Multiple /var/folders/38/hws16c9j0qbfm9_r3t900slw0000gn/T/__seg__mask__ True

should be:
/Library/Frameworks/Python.framework/Versions/3.9/bin/python3[i] "/Users/efe/Library/Application Support/GIMP/2.10/plug-ins/seganybridge.py" [/i]vit_h /Users/me/DeepInDepth/SAM/sam_vit_h_4b8939.pth /var/folders/38/hws16c9j0qbfm9_r3t900slw0000gn/T/__seg__7Szrd4.png Auto Multiple /var/folders/38/hws16c9j0qbfm9_r3t900slw0000gn/T/__seg__mask__ True

Even with that change the seganybridge.py does not start. Bizarre!

[khemadeva]

(11-08-2023, 09:42 AM)LeafAtWood Wrote:

(11-07-2023, 03:49 AM)LeafAtWood Wrote: Thanks khemadeva for the plugin, though I am having trouble registering it with my GIMP 2.10 on macOS 10.15.7. I have run your install validation script (/path/to/python3/python ./seganybridge.py vit_h /path/to/checkpoint/model/sam_vit_h_4b8939.pth) and got a Success. I have put the seganybridge.py and seganyplugin.py in the /Users/me/Library/Application Support/GIMP/2.10/plug-ins repository. I also changed the mode so that these files are executable (chmod 777). Made sure I restarted GIMP. I also made sure SAM was properly installled by running the script amg.py from the SAM github and generated the segmentation masks. So seems my only trouble is registering your plugin with GIMP. Do you have any idea why it is not registering with GIMP and would you know of any way to check/debug the registration process ?

I have relayed the news on my Deep_in_Depth twitter channel : https://twitter.com/Deep_In_Depth/status...9651147837

Fixed the "registration issue" by just adding the following line to the top of each file seganybridge.py and seganyplugin.py
#!/usr/bin/env python

There also seem to be a problem with the default plug-ins path on macOS: /Users/me/Library/Application Support/GIMP/2.10/plug-ins   due to the space in the path 

When the seganyplugin.py calls the seganybridge.py it looks like this:
/Library/Frameworks/Python.framework/Versions/3.9/bin/python3 /Users/efe/Library/Application Support/GIMP/2.10/plug-ins/seganybridge.py vit_h /Users/me/DeepInDepth/SAM/sam_vit_h_4b8939.pth /var/folders/38/hws16c9j0qbfm9_r3t900slw0000gn/T/__seg__7Szrd4.png Auto Multiple /var/folders/38/hws16c9j0qbfm9_r3t900slw0000gn/T/__seg__mask__ True

should be:
/Library/Frameworks/Python.framework/Versions/3.9/bin/python3[i] "/Users/efe/Library/Application Support/GIMP/2.10/plug-ins/seganybridge.py" [/i]vit_h /Users/me/DeepInDepth/SAM/sam_vit_h_4b8939.pth /var/folders/38/hws16c9j0qbfm9_r3t900slw0000gn/T/__seg__7Szrd4.png Auto Multiple /var/folders/38/hws16c9j0qbfm9_r3t900slw0000gn/T/__seg__mask__ True

Even with that change the seganybridge.py does not start. Bizarre!

Have you been able to run the plugin? Do you see any errors on console? Please let me know and I will look into the issue.

[khemadeva]

(11-08-2023, 09:42 AM)LeafAtWood Wrote:

(11-07-2023, 03:49 AM)LeafAtWood Wrote: Thanks khemadeva for the plugin, though I am having trouble registering it with my GIMP 2.10 on macOS 10.15.7. I have run your install validation script (/path/to/python3/python ./seganybridge.py vit_h /path/to/checkpoint/model/sam_vit_h_4b8939.pth) and got a Success. I have put the seganybridge.py and seganyplugin.py in the /Users/me/Library/Application Support/GIMP/2.10/plug-ins repository. I also changed the mode so that these files are executable (chmod 777). Made sure I restarted GIMP. I also made sure SAM was properly installled by running the script amg.py from the SAM github and generated the segmentation masks. So seems my only trouble is registering your plugin with GIMP. Do you have any idea why it is not registering with GIMP and would you know of any way to check/debug the registration process ?

I have relayed the news on my Deep_in_Depth twitter channel : https://twitter.com/Deep_In_Depth/status...9651147837

Fixed the "registration issue" by just adding the following line to the top of each file seganybridge.py and seganyplugin.py
#!/usr/bin/env python

There also seem to be a problem with the default plug-ins path on macOS: /Users/me/Library/Application Support/GIMP/2.10/plug-ins   due to the space in the path 

When the seganyplugin.py calls the seganybridge.py it looks like this:
/Library/Frameworks/Python.framework/Versions/3.9/bin/python3 /Users/efe/Library/Application Support/GIMP/2.10/plug-ins/seganybridge.py vit_h /Users/me/DeepInDepth/SAM/sam_vit_h_4b8939.pth /var/folders/38/hws16c9j0qbfm9_r3t900slw0000gn/T/__seg__7Szrd4.png Auto Multiple /var/folders/38/hws16c9j0qbfm9_r3t900slw0000gn/T/__seg__mask__ True

should be:
/Library/Frameworks/Python.framework/Versions/3.9/bin/python3[i] "/Users/efe/Library/Application Support/GIMP/2.10/plug-ins/seganybridge.py" [/i]vit_h /Users/me/DeepInDepth/SAM/sam_vit_h_4b8939.pth /var/folders/38/hws16c9j0qbfm9_r3t900slw0000gn/T/__seg__7Szrd4.png Auto Multiple /var/folders/38/hws16c9j0qbfm9_r3t900slw0000gn/T/__seg__mask__ True

Even with that change the seganybridge.py does not start. Bizarre!

Hello @LeafAtWood I have modified the script on github to accept spaces in the directory, please use the new script. Here are a couple other things to note 1) Make sure the segment-anything folder you created with git clone is in the PYTHONPATH 2) Check the python version from Python-Fu console, it should be 2.7.x. The script will run with only python2 since, to my knowledge, GIMP 2.10 has not yet upgraded to python3. Please let me know if you still face any issues.

[tec0]

(10-07-2023, 09:33 AM)khemadeva Wrote: I have created a plugin, which allows users to isolate objects within a raster image inside GIMP using Meta Segment Anything.
Please feel free to download and use if you find it useful.

Here are the links:

Github Repo

Video Tutorial

Hi, I have followed all the steps and got a "success!!" from the test, but there is no correct result when using the plugin, there are no layers being created, and shows the following errors:
GEGL-Message: 00:49:01.515: Module 'C:\Program Files\GIMP 2\lib\gegl-0.4\exr-save.dll' load error: 'C:\Program Files\GIMP 2\lib\gegl-0.4\exr-save.dll': The specified module could not be found.

GEGL-Message: 00:49:01.519: Module 'C:\Program Files\GIMP 2\lib\gegl-0.4\exr-load.dll' load error: 'C:\Program Files\GIMP 2\lib\gegl-0.4\exr-load.dll': The specified module could not be found.

2023-11-29 00:49:01 - DEBUG - pythonPath: C:\Users\s\AppData\Local\Programs\Python\Python310\python.exe

C:\Users\s\AppData\Roaming\GIMP\2.10\plug-ins\[color=var(--yt-endpoint-color,var(--yt-spec-call-to-action))]seganyplugin.py:433[/color]: Warning: value "0" of type 'guint' is invalid or out of range for property 'n-rows' of type 'guint'

 table = gtk.Table(0, 0, False)

C:\Users\seana\AppData\Roaming\GIMP\2.10\plug-ins\[color=var(--yt-endpoint-color,var(--yt-spec-call-to-action))]seganyplugin.py:433[/color]: Warning: value "0" of type 'guint' is invalid or out of range for property 'n-columns' of type 'guint'

 table = gtk.Table(0, 0, False)

Could you offer any help on this?

[LeafAtWood]

(11-20-2023, 05:45 AM)khemadeva Wrote:

(11-08-2023, 09:42 AM)LeafAtWood Wrote:

(11-07-2023, 03:49 AM)LeafAtWood Wrote: Thanks khemadeva for the plugin, though I am having trouble registering it with my GIMP 2.10 on macOS 10.15.7. I have run your install validation script (/path/to/python3/python ./seganybridge.py vit_h /path/to/checkpoint/model/sam_vit_h_4b8939.pth) and got a Success. I have put the seganybridge.py and seganyplugin.py in the /Users/me/Library/Application Support/GIMP/2.10/plug-ins repository. I also changed the mode so that these files are executable (chmod 777). Made sure I restarted GIMP. I also made sure SAM was properly installled by running the script amg.py from the SAM github and generated the segmentation masks. So seems my only trouble is registering your plugin with GIMP. Do you have any idea why it is not registering with GIMP and would you know of any way to check/debug the registration process ?

I have relayed the news on my Deep_in_Depth twitter channel : https://twitter.com/Deep_In_Depth/status...9651147837

Fixed the "registration issue" by just adding the following line to the top of each file seganybridge.py and seganyplugin.py
#!/usr/bin/env python

There also seem to be a problem with the default plug-ins path on macOS: /Users/me/Library/Application Support/GIMP/2.10/plug-ins   due to the space in the path 

When the seganyplugin.py calls the seganybridge.py it looks like this:
/Library/Frameworks/Python.framework/Versions/3.9/bin/python3 /Users/efe/Library/Application Support/GIMP/2.10/plug-ins/seganybridge.py vit_h /Users/me/DeepInDepth/SAM/sam_vit_h_4b8939.pth /var/folders/38/hws16c9j0qbfm9_r3t900slw0000gn/T/__seg__7Szrd4.png Auto Multiple /var/folders/38/hws16c9j0qbfm9_r3t900slw0000gn/T/__seg__mask__ True

should be:
/Library/Frameworks/Python.framework/Versions/3.9/bin/python3[i] "/Users/efe/Library/Application Support/GIMP/2.10/plug-ins/seganybridge.py" [/i]vit_h /Users/me/DeepInDepth/SAM/sam_vit_h_4b8939.pth /var/folders/38/hws16c9j0qbfm9_r3t900slw0000gn/T/__seg__7Szrd4.png Auto Multiple /var/folders/38/hws16c9j0qbfm9_r3t900slw0000gn/T/__seg__mask__ True

Even with that change the seganybridge.py does not start. Bizarre!

Hello @LeafAtWood I have modified the script on github to accept spaces in the directory, please use the new script. Here are a couple other things to note 1) Make sure the segment-anything folder you created with git clone is in the PYTHONPATH 2) Check the python version from Python-Fu console, it should be 2.7.x. The script will run with only python2 since, to my knowledge, GIMP 2.10 has not yet upgraded to python3. Please let me know if you still face any issues.

Hello @khemadeva,
Sorry for the late answer. Didn't activate the "Default Thread Subscription Mode:". So didn't see you answer.
Thanks for the update and the advices. Will check the new version ASAP and let you know how it works.

[LeafAtWood]

Hello khemadeva

After installing your new version of the code, I sent the logging output to a file.
There is still an issue, but as you can see below it is a more "basic" issue with numpy (multiarray) import in the python 2.7 in python-fu module.

Not sure how to fix that. "import numpy" works fine in the interactive python-fu console.

The file /Applications/GIMP.app/Contents/Resources/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/_multiarray_umath.so is present. 

Messages put in the logging file

Quote:2023-12-17 07:08:03 - DEBUG - pythonPath: /Library/Frameworks/Python.framework/Versions/3.9/bin/python3.9
2023-12-17 07:08:11 - INFO - Running command: /Library/Frameworks/Python.framework/Versions/3.9/bin/python3.9 /Users/efe/Library/Application Support/GIMP/2.10/plug-ins/seganybridge.py vit_h /Users/efe/DeepInDepth/SAM/sam_vit_h_4b8939.pth /var/folders/38/hws16c9j0qbfm9_r3t900slw0000gn/T/__seg__o4y3wV.png Box Single /var/folders/38/hws16c9j0qbfm9_r3t900slw0000gn/T/__seg__mask__ True sel_place_holder 223,156,871,618
2023-12-17 07:08:16 - ERROR - Command failed with the following error:
 Traceback (most recent call last):
  File "/Applications/GIMP.app/Contents/Resources/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/__init__.py", line 40, in <module>
    from . import multiarray
  File "/Applications/GIMP.app/Contents/Resources/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/multiarray.py", line 13, in <module>
    from . import overrides
  File "/Applications/GIMP.app/Contents/Resources/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/overrides.py", line 6, in <module>
    from numpy.core._multiarray_umath import (
ImportError: dlopen(/Applications/GIMP.app/Contents/Resources/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/_multiarray_umath.so, 2): Library not loaded: @rpath/lib/libopenblas-r1.dylib
  Referenced from: /Applications/GIMP.app/Contents/Resources/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/_multiarray_umath.so
  Reason: image not found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/efe/Library/Application Support/GIMP/2.10/plug-ins/seganybridge.py", line 25, in <module>
    import torch
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/torch/__init__.py", line 1120, in <module>
    from .storage import _StorageBase, TypedStorage, _LegacyStorage, UntypedStorage, _warn_typed_storage_removal
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/torch/storage.py", line 14, in <module>
    import numpy as np
  File "/Applications/GIMP.app/Contents/Resources/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/__init__.py", line 142, in <module>
    from . import core
  File "/Applications/GIMP.app/Contents/Resources/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/__init__.py", line 71, in <module>
    raise ImportError(msg)
ImportError: 

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the multiarray numpy extension module failed.  Most
likely you are trying to import a failed build of numpy.
Here is how to proceed:
- If you're working with a numpy git repository, try git clean -xdf
  (removes all files not under version control) and rebuild numpy.
- If you are simply trying to use the numpy version that you have installed:
  your installation is broken - please reinstall numpy.
- If you have already reinstalled and that did not fix the problem, then:
  1. Check that you are using the Python you expect (you're using /Library/Frameworks/Python.framework/Versions/3.9/bin/python3.9),
     and that you have no directories in your PATH or PYTHONPATH that can
     interfere with the Python and numpy versions you're trying to use.
  2. If (1) looks fine, you can open a new issue at
     https://github.com/numpy/numpy/issues.  Please include details on:
     - how you installed Python
     - how you installed numpy
     - your operating system
     - whether or not you have multiple versions of Python installed
     - if you built from source, your compiler versions and ideally a build log

     Note: this error has many possible causes, so please don't comment on
     an existing issue about this - open a new one instead.

Original error was: dlopen(/Applications/GIMP.app/Contents/Resources/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/_multiarray_umath.so, 2): Library not loaded: @rpath/lib/libopenblas-r1.dylib
  Referenced from: /Applications/GIMP.app/Contents/Resources/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/_multiarray_umath.so
  Reason: image not found

[BeaverGEGLFreak]

My plugins were briefly brought up in this thread on page 1 in what appears to be an unrelated warning that I get too. I Believe that is caused by someone elses python GEGL plugin and it does no harm. Just please let me know if it turns out my GEGL plugins conflict with this plugin. I doubt that is the case though.