Gimp-Forum.net
GIMP not responding on high file sizes. - Printable Version

+- Gimp-Forum.net (https://www.gimp-forum.net)
+-- Forum: GIMP (https://www.gimp-forum.net/Forum-GIMP)
+--- Forum: General questions (https://www.gimp-forum.net/Forum-General-questions)
+--- Thread: GIMP not responding on high file sizes. (/Thread-GIMP-not-responding-on-high-file-sizes)

Pages: 1 2

GIMP not responding on high file sizes. - techie1484 - 05-15-2019

Greetings,

I created an animation GIF which has 500 frames.It is taking too long to unoptimize and use interleave layer plugin. Even after waiting to complete the plugin execution, When trying to save it, GIMP freezes with message "not responding".

I have enough space on my disk to save the file. The final file size is about 1.5 GB.

Any pointers on how to improve the performance would be helpful.

RE: GIMP not responding on high file sizes. - Ofnuts - 05-15-2019

1.5GB can takes a good minute to write to disk (or more...). Do you see the file size increasing on the disk?

Also, what is your tile cache set to? (Edit>Preferences>System resources>Tile cache size). If not set high enough, Gimp will self-limit its RAM usage and this will slow it down.

RE: GIMP not responding on high file sizes. - rich2005 - 05-15-2019

If Ofnuts suggestion does not improve performance.

You could partly abandon Gimp and use command line applications for the unoptimise part, that might help (or maybe not Wink ) It might save a little time only partially using Gimp.

This old application gifsicle https://www.lcdf.org/gifsicle/ can unoptimise screenshot: https://i.imgur.com/eBFat0O.jpg

Code:
gifsicle.exe -U input.gif -o output.gif

Another approach might be split the gif up into separate frames using ImageMagic (IM) http://www.imagemagick.org This will unoptimise and render each frame as a png https://i.imgur.com/shRQcuJ.jpg

Code:
magick convert -coalesce input.gif output%05d.png

Then it depends on what you are interlacing? If an overlay of a single logo, might be easy, maybe using Gimp BIMP plugin on a folder full of files. Another animation to interlace, not so easy. IM has a full (confusing) page of various procedures. http://www.imagemagick.org/Usage/anim_mods/#merging

You can also use IM to easily combine back to an animation: https://i.imgur.com/VaZvQf9.jpg

Code:
magick convert -delay 20 -loop 0 *png output.gif


RE: GIMP not responding on high file sizes. - Ofnuts - 05-15-2019

This said, given the size, using the more recent WebP format (supported in Gimp 2.10) could be a good idea.

RE: GIMP not responding on high file sizes. - techie1484 - 05-15-2019

(05-15-2019, 08:01 AM)Ofnuts Wrote:  If not set high enough, Gimp will self-limit its RAM usage and this will slow it down.

How much higher does it need to be set???

My RAM is 4 GB.

In the preferences dialog of GIMP,Tile Cache Size is set to 2048484 KB.

I also observed some oddity in disk space consumption while GIMP was executing plugins.

I found it odd that the disk space consumption was 100 % in Task Manager even when there were no applications running(besides the background processes).

Initially, the issue was due to Windows updates causing this consumption due to heavy I/O on a file that Windows updates access.(forgot to copy down the file name).So i installed updates from control panel and restarted my PC. Now, the disk consumption was reduced to single digits(in percentage) when there were no applications running.

After starting GIMP, I still was able to observe oddities in disk consumption when performing operations in GIMP.

GIMP would freeze with the message  "not responding" when disk consumption hits "100 %". And there were no other applications running besides Task Manager and Resource Monitor.

Task manager screenshot:
[Image: taskmanager.png]

As we can see, Disk consumption is at 99 % when only GIMP is running.

Screenshot of Resource Monitor:

[Image: Resourcemonitor.png]

Screenshot in full size:

https://ibb.co/2d1Cw1n

Screenshot 2 of Resource Monitor:

[Image: Resourcemonitor-2.png]

Screenshot 2 in full size:

https://ibb.co/hL0Gtzr

Screenshot 3 of Resource Monitor which has write operations:

[Image: After-start-of-write.png]

Screenshot of resource monitor with write operations in full size:

https://ibb.co/C7YBg0X

I looked up on Pagefile.sys and it happened to be a buffer on disk if RAM is fully incapacitated. And 4 GB RAM is quite big.

http://m.majorgeeks.com/content/page/everything_you_need_to_know_about_pagefile_sys.html

Since the working directory of GIMP on my PC is C:, I have deleted files and freed space in C:. But still GIMP displays "not responding" and consumes 100 % disk while executing GIMP plugins.

Is the issue because of plugin closed/terminated in an inconsistent state???Cuz i think 100 % disk consumption for one application seems odd.

@rich2005 :

Thank you for your suggestions. As i inferred above, if the issue is not with disk consumption when executing plugins in GIMP, We will go command line if all other options are exhausted.

RE: GIMP not responding on high file sizes. - rich2005 - 05-15-2019

(05-15-2019, 09:18 AM)Ofnuts Wrote: This said, given the size, using the more recent WebP format (supported in Gimp 2.10) could be a good idea.

@Ofnuts

Re webp ... Ouch.... A small (320 x 220)  video cartoon broken to 580  frames as jpegs.
In Gimp as layers 124 MB  in memory 
Render as a gif = 12 MB Optimised as a gif = 10.5 MB, either takes a few seconds

Render as an animated webp with a quality about 60 and file size down to 2.5 MB  but took 30 minutes Wink

Batch converting jpeg -> webp with IM is pretty quick but I am  struggling with command line webpmux (horrible syntax)  to make an animation.

There is gif2webp which is reasonably quick but then stuck with the 256 colours.

RE: GIMP not responding on high file sizes. - Ofnuts - 05-15-2019

Looks like you have a fairly small system for what you want to do.

Gimp tells you how much memory it uses in the status bar at the bottom, next to the name of the current layer, when this area isn't used to display a message (so move the mouse outside of the image display, for instance to the toolbox).

IMHO Gimp is using much more than 2GB and you have two overlapping problems:
  • The 2GB of RAM for the tile cache are above what the system can really provide (what else is running?) so even before Gimp tries to use its own disk-swapping mechanism, some of its memory is swapped out by the system (the pagefile.sys file you see in the monitor panel)
  • At some point Gimp has more than 2GB of data, so starts its own swapping mechanism  in addition to what the system already does.
So you get twice the disk I/O: Gimp paging and system paging. Two ways to improve on this:

  1. set the tile cache to something real big (8GB or more) Gimp won't do its own swapping (until you reach that limit...) and you will onlt have system swapping
  2. set the tile cache to something small (down to 1GB, and then try increasing if see free RAM) so you system never swaps and Gimp doesn its own swapping
Other things to do:

  • reduce RAM usage elsewhere in the system, stop any non necessary processes (web browsers are great memory hogs)
  • reduce RAM usage in Gimp (use 8-bit precision, use a small undo stack...)



RE: GIMP not responding on high file sizes. - techie1484 - 05-24-2019

(05-15-2019, 06:31 PM)Ofnuts Wrote:
  1. set the tile cache to something real big (8GB or more) 

Tried this but no luck.

I tried other things like uninstalling some apps and delete files.Apparently , Windows updates seem to be the most disruptive but once they were installed , my system performance was back to normal.

I have attempted this kind of GIF before but not exceeding 220 frames.

https://twitter.com/Techie1484/status/1106781076930723845

Sorry, I don't have the file on my PC as i had to delete them to free space. The above was optimized for social media after uploading to GIPHY, downloading it to my PC and uploaded on to twitter. Cuz twitter has file size limits. The above is less than 220 frames.

When i download movies from the web, they have all been 720 P resolution and the final file size doesn't exceed 850 MB. 

The static layer i use on the animation is less than 2 MB.

But the final file size of 500 frame animation is 1.5 GB.

I don't know of compression methods on movie type files which Media players are compatible with which may be the reduced final file size being less than 850 MB.

I guess i will jus have to stick with less than 230 frames.

RE: GIMP not responding on high file sizes. - Ofnuts - 05-24-2019

May be you should try other formats. WebP is a bit like GIF but the frames can be lossly compressed as with JPEGs. It is supported in Gimp 2.10 and support by all browsers. The final result can be much smaller. There is also its cousin the WebM format, which is more video-like.

AFAIK Gify transforms the animation in video anyway.

PS: For 1.5GB I can have a 720p copy of any StarWars epidsode.

RE: GIMP not responding on high file sizes. - rich2005 - 05-24-2019

IMHO The problem is going to be lack of resources somewhere along the line, probably memory.

720p(rogressive) is 1280x720 frame size so making a 500 frame video that size.

I record in a comparatively loose compressed avi. (uncompressed, runs to Gb's - not used that for 20 years)
That clip is  8.6 MB. I export in divx(xvid) format for youtube 1.8 MB but that is a finished format not really suitable for video editing, and does not loop for web use.
  
I split that up using ffmpeg into 500 png images. Using my laptop which is very average these days, i5 2.4GHz  CPU / 8 GB memory

500 frames load (2 min -30sec). Shows as 1.4 MB in memory https://i.imgur.com/xgYC7oa.jpg I can export as a gif animation full frame replace that took 2min-30 as well. gif file size - whopping 172 MB The frames do optimise in GIMP and export as a gif in combine mode, file size 2.8 MB Both these re-open in Gimp ok.

On to webp format. These do loop, file size can be better or equal to gif, hopefully better colour. Gimp will make a animated webp, from that earlier experience, very slow. So start it going, take the dog for a walk, have cups of coffee.

Caveat: I use linux but AFAIK there are Windows versions. 
Other ways: webp is a Google thing and there are a few tools, primarily for adding a few images as an animation. However there is one gif2webp which does what it says.  Gets a webp from a gif 5 to 6 MB in this case.

A better way is use those original separate png's with ffmpeg to render the webp video.  Size is smaller, 3 MB. Unfortunately the webp format is still not universal, not going to display here or imgur, so a little video of the results for comparison





direct link https://youtu.be/C1Dwdb_9Qdo  duration  two and a half minute