Accelerate processing of Gimp scripts

[Gimpnik]

State:
   Gimp: 2.10.32
   OS: Win7 x32-Bit

Processing of tool scripts is unnecessarily slowed down:
Currently it is the case that when batch processing is performed by Gimp via external batch script (tested here especially under Windows), it is noticeable that gimp.exe starts in a new process instance each time, which unnecessarily slows down the repetition of scripts or execution of further commands, since Gimp first uploads all its modules each time - or even worse for each command line in the script. An unnecessary nonsense.
So it would be much more superior and faster, if there would be a parameter here to let gimp.exe load once into RAM and stay there until it is not explicitly removed from there something by (gimp-quit 0).
Under "gimp-2.10.exe --help" however no further parameters are listed at present, which force the execution of the Gimp console in already running instance.
 
And concretely on the example: A following processing under Windows leads currently per gimp.exe command line a delay of around 5 s, so with next following both command line per gimp.exe thus at least (2* 5 =) ~ 10 s are wasted:

Code:

PushD %~dp0
 
set "gimp=%ProgramFiles%\GIMP 2\bin\gimp-2.10.exe"
set "FileExtens=PNG"
 
"%gimp%" -i -b "(batch-crop-to-content \"*.%FileExtens%\")"
"%gimp%" -i -b "(scale \"*.%FileExtens%\")"
"%gimp%" -i -b "(rotate \"*.%FileExtens%\")"

 
(In addition there is the closing of Gimp, which is also somewhat delayed).

Suggestions for improvement?

[Ofnuts]

If Gimp is already started when you call it from a shell, then the arguments are passed to the already loaded instance. But of course each "gimp" command in your script will run bow nearly instantaneously because it returns before the work is done, so if each needs the result of the previous one that won't do it. But the first thing to do to improve performance is to string your commands into a single script, and so call gimp only once.

You can also use ‑‑new‑instance to have separate instances of Gimp and make them run in parallel (I don't know how to do that in Windows) so if you process several files you can use all your processors (Gimp is still essentially single-thread).

There are also a couple of options to the Gimp command (‑‑no‑data, ‑‑no‑fonts) to tell Gimp to not load fonts, patterns, etc... that can speedup the startup.

[Gimpnik]

Thanks

(09-25-2022, 11:57 AM)Ofnuts Wrote: If Gimp is already started when you call it from a shell, then the arguments are passed to the already loaded instance.

Unfortunately, this is not the case for me on Windows: regardless of whether the tool is already running in the foreground, unfortunately a new separate process instance is currently always executed.

(09-25-2022, 11:57 AM)Ofnuts Wrote: But of course each "gimp" command in your script will run bow nearly instantaneously because it returns before the work is done, so if each needs the result of the previous one that won't do it.

No, in "batches" is this behavior as you described not usual and Windows Batch Scripts not either unless each gimp command line is executed on the beginning as follows to force a new process instance:

Code:

START "" "%gimp%" …

By default, batch processing, as the name implies, waits unless starting processes in separate instances is enforced (only then does it immediately proceed to the next line).

But you can also force to wait if the default behavior the other way around:

Code:

START "" /WAIT "%gimp%" …

NOTES: Each additional Gimp command line will be executed only after gimp.exe console is closed and in Windows cmd.exe console the batch process is continued (in case by pressing N key for No - to do not cancel batch operation)

(09-25-2022, 11:57 AM)Ofnuts Wrote: But the first thing to do to improve performance is to string your commands into a single script, and so call gimp only once.

You certainly mean here "into a single command line", because considered as a "file" it is a single script - in my case from this point of view it is the same script, but in another context it is several, depending on how it is viewed.

Of course it is possible but syntax-technically and clearly disastrous, especially because many arguments per command line will follow, which have not been implemented here yet:

Code:

"%gimp%" -i -b "(BatchCropToContent \"*.%FileExtens%\")"
"%gimp%" -i -b "(scale \"*.%FileExtens%\")" "Arg1" "Arg2" "Arg3"
"%gimp%" -i -b "(rotate \"*.%FileExtens%\")" "Arg1" "Arg2" "Arg3"

(09-25-2022, 11:57 AM)Ofnuts Wrote: You can also use   new instance to have separate instances of Gimp and make them run in parallel (I don't know how to do that in Windows) so if you process several files you can use all your processors (Gimp is still essentially single-thread).

All right, but that is not the described problem here. Here now it doesn't matter that the tool is not multitasking on a single instance.
Indirect multitasking (with more single processes) is possible, but then unfortunately not for the same pictures.

(09-25-2022, 11:57 AM)Ofnuts Wrote: There are also a couple of options to the Gimp command …

Yes, I see, but this does not help by far to solve that main problem described here.

Nevertheless thank you very much for joining!!!

[Ofnuts]

(09-25-2022, 02:01 PM)Gimpnik Wrote: Thanks

(09-25-2022, 11:57 AM)Ofnuts Wrote: If Gimp is already started when you call it from a shell, then the arguments are passed to the already loaded instance.

Unfortunately, this is not the case for me on Windows: regardless of whether the tool is already running in the foreground, unfortunately a new separate process instance is currently always executed.

Could be a gimp-console vs gimp thing.

(09-25-2022, 02:01 PM)Gimpnik Wrote:

(09-25-2022, 11:57 AM)Ofnuts Wrote: But of course each "gimp" command in your script will run bow nearly instantaneously because it returns before the work is done, so if each needs the result of the previous one that won't do it.

No, in "batches" is this behavior as you described not usual and Windows Batch Scripts not either unless each gimp command line is executed on the beginning as follows to force a new process instance:

Code:

START "" "%gimp%" …

By default, batch processing, as the name implies, waits unless starting processes in separate instances is enforced (only then does it immediately proceed to the next line).

But you can also force to wait if the default behavior the other way around:

Code:

START "" /WAIT "%gimp%" …

NOTES: Each additional Gimp command line will be executed only after gimp.exe console is closed and in Windows cmd.exe console the batch process is continued (in case by pressing N key for No - to do not cancel batch operation)

Yes, but... when you call the gimp command, it checks if there is an instance running already (unless you use the --new-instance parameter) and if it finds one it passes it the command arguments and the just started instance exits immediately. This is not the same as the started instance continuing in the background (as with START).  Since your batch file waits for the instance it started, it sees that instance terminate and continues. Unless, as above,  a gimp-console vs gimp thing.

(09-25-2022, 02:01 PM)Gimpnik Wrote:

(09-25-2022, 11:57 AM)Ofnuts Wrote: But the first thing to do to improve performance is to string your commands into a single script, and so call gimp only once.

You certainly mean here "into a single command line", because considered as a "file" it is a single script - in my case from this point of view it is the same script, but in another context it is several, depending on how it is viewed.

Of course it is possible but syntax-technically and clearly disastrous, especially because many arguments per command line will follow, which have not been implemented here yet:

Code:

"%gimp%" -i -b "(BatchCropToContent \"*.%FileExtens%\")"
"%gimp%" -i -b "(scale \"*.%FileExtens%\")" "Arg1" "Arg2" "Arg3"
"%gimp%" -i -b "(rotate \"*.%FileExtens%\")" "Arg1" "Arg2" "Arg3"

Simplicity and performance are not always synonyms. But if you make that a true script, then you can possibly compute some of the scale and rotation arguments for some well picked initial arguments, so you likely need fewer parameters than you think.

And btw, while we are at it, rotation and scaling introduce some blur. If you do two operations (one scaling, or rotation), you blur twice.  There are gimp-drawable-transform-2d and gimp-drawable-transform-2d-default functions that do both in one operation so you only blur once.

[Gimpnik]

Ofnuts Wrote:Could be a gimp-console vs gimp thing.

This then gives hope that the developers are at least on the right track. So it remains the correct implementation to find the error / the cause why it does not work in Windows.

Ofnuts Wrote:Yes, but... when you call the gimp command, it checks if there is an instance running already (unless you use the --new-instance parameter) and if it finds one it passes it the command arguments …

And here the same as before: It gives hope and it remains to be seen until the bug is fixed in Gimp for Windows platform.

Ofnuts Wrote:… and the just started instance exits immediately

Something is wrong here, because just such behavior would then explain the essential problem here.
It makes no sense to terminate an already running instance without being asked, and even more so if there is a possibility to terminate it on demand, e.g. via (gimp-quit 0).
And just to be on the safe side, I'll mention it here again: no, unfortunately this is not currently the case in Windows. When executing a Gimp command, an already running instance is not taken into account to feed it with the command, but a new process instance is started, which makes the whole thing very slow.

Ofnuts Wrote:This is not the same as the started instance continuing in the background (as with START). Since your batch file waits for the instance it started, it sees that instance terminate and continues. Unless, as above, a gimp-console vs gimp thing.

I'm afraid there's a lot of confusion here: As described above it is processed especially in this case classically line by line and without START "" gimp … (in separated process instances), and yet, a separate process instance is executed for each Gimp command line.

Ofnuts Wrote:Simplicity and performance are not always synonyms.

That is true and I fully agree, however, I did not claim the opposite here.

Ofnuts Wrote:But if you make that a true script, then you can possibly compute some of the scale and rotation arguments for some well picked initial arguments, so you likely need fewer parameters than you think.

That may be already so. I just wanted to point out that the whole structure should remain clear, because some arguments and other things will follow and the whole thing should remain clear and easy to maintain. Otherwise it is an unattractive nesting. But all this is secondary, because in case of need it would be possible to create an intermediate procedure, which summarizes everything to one line of code in the end, to run only one Gimp instance.

So the primary thing now would be to find out why Gimp in Windows does not consider an already running process instance, but always executes a new instance.

Ofnuts Wrote:And btw, while we are at it, rotation and scaling introduce some blur. If you do two operations (one scaling, or rotation), you blur twice.  There are gimp-drawable-transform-2d and gimp-drawable-transform-2d-default functions that do both in one operation so you only blur once.

That's fine to know. Many thanks for this valuable tip! Very fine!