-Xmx ignored by GPT?

Optical Toolbox
1

Hi,

I am trying to deal with an S2 Idepix processing graph that attempts to use too much memory when processing a larger area (still a subset of the overall tile) and is killed.

My understanding is that I should set the maximum memory using -Xmx in the gpt.vmoptions. When I set -Xmx to 12G on my 32G linux machine the GPT graph uses up all the available memory (not just 12G) up to 32G and is then killed.

I can remedy the issue by using a machine with more RAM but I am confused about why the -Xmx parameter is not limiting the memory usage and in general seems to have little effect?

Does anyone have any ideas? Am I using the gpt.vmoptions incorrectly?

Thanks

James

2

I’d suggest to split your processing into smaller chunks. It can be more time-consuming but might help to stay within your memory limits.

3

Do you use NetCDF format in your processing.
SNAP uses the native NetCDF library, and this uses additional memory outside of the Java VM. Not controllable by the Xmx option.

EOMasters_icon_60 Marco from EOMasters - Mastering Earth Observation

1 Like
Process killed when writing to NetCDF4-CF
4

In certain cases this is the expected behavior that you are observing. This is mainly because the -Xmx parameter can limit only the Java heap memory and not the total memory. To better manage this issue, you may need to monitor both heap and non-heap memory usage. When you are allocating -Xmx=12G, this will restrict only the heap size to 12 GB. Still, there is a possibility for the application to take additional memory outside the heap, such as native memory, buffers, thread stacks, and others like memory-mapped files, etc. As a result, the overall memory usage of the GPT process may grow beyond the specified heap limit and reach the system’s available memory. You will have to verify if this gpt.vmoptions file is being correctly picked up. Suppose if the configuration is not applied properly, the JVM may ignore the specified -Xmx value. To understand this better and get proper usage steps on how to set the heap size to your application, you can refer to this blog, Sizing Your Heap Correctly: Understanding -Xms and -Xmx.