S1 TOPS Coregistration error - Java heap space error?


#1

Dear all,

i am experiencing following problems when processing S1 TOPS Coregistration with ESD.
my S1 dataset located over Iceland (acquisition dates 04.06.17 and 16.06.17 recently downloaded from scihub, we use the external DEM for back geocoding)
First i thought it was the data that was too new (i firstly started a few days ago processing the scenes acquired on 22.06.17 and 10.06.17) and snap couldn’t find precise orbit file at that time yet that’s why error (by looking at the log file)
Then i switched to the older dataset said above and also increase the heap space size -Xms8g (see below) max heap size have been set to -Xmx88g, still i got the below error.
could someone help solving this problem?
i run snap on the remote Linux server (128 gb RAM) from my PC.
Linux OS is Centos 7.0, Snap latest version 5.0 andS1 Toobox version 5.0.5 were used
thank you very much in advance.


[username@linux-server ~]$ snap -J-Xms8g
Entire image is outside of SRTM valid area.
Please use another DEM.
Entire image is outside of SRTM valid area.
Please use another DEM.
Entire image is outside of SRTM valid area.
Please use another DEM.
Entire image is outside of SRTM valid area.
Please use another DEM.
Entire image is outside of SRTM valid area.
Please use another DEM.
Entire image is outside of SRTM valid area.
Please use another DEM.
Entire image is outside of SRTM valid area.
Please use another DEM.
java.lang.OutOfMemoryError: Java heap space
Dumping heap to /home/jirathana/.snap/system/var/log/heapdump.hprof …
Java HotSpot™ 64-Bit Server VM warning: record is too large
Java HotSpot™ 64-Bit Server VM warning: record is too large
Java HotSpot™ 64-Bit Server VM warning: record is too large
Heap dump file created [53930262169 bytes in 198.193 secs]
Java heap space
Java heap space
java.lang.NullPointerException
java.lang.NullPointerException
java.lang.StackOverflowError
java.lang.StackOverflowError
java.lang.NullPointerException
java.lang.NullPointerException
Java heap spacejava.lang.StackOverflowError
java.lang.StackOverflowError
java.lang.StackOverflowError
java.lang.StackOverflowError
java.lang.NullPointerException
java.lang.NullPointerException
java.lang.NullPointerException


#2

did you allow SNAP to use the 128 GB of RAM?


#3

@ABraun

i’m not quite sure how many GB of RAM are allowed to use by SNAP on my linux server. And i couldn’t find snap.conf file in the ‘etc’ "folder at all. Is it a kind of ‘hidden’ file in the directory?

one more question: is it possible to coregister S1 (SLC) subset images with this operator> S1 TOPS Coregistration?
Have you got this experience?

thank you for the help
regards,


#4

You can also accomplish this under Tools > Options. If you leave some RAM for the Operating system it is fine. I’d try 112GB for a start.

Yes. Please have a look here:


#5

I have the same heap error issue and have tried to apply the fix using the Tools->Options but the VM window cannot be edited in my version of SNAP. I don’t have a snap.conf file either or I would try that.

Any further advice will be appreciated, thanks.


#6

even if i subset the image (reduced 50% of size), i still got the java heap space error.
BTW i haven’t seen the bar used to select the Bursts as well thanks to @ABraun :slight_smile: a really good Tipp.

at Tool > options i can’t modify VM parameters as well (they are in grey and faded out), although my admin said that i do have all rights to read and write the file.

thank you for all advices and for further advices! i really would like to get the things solved…


#7

then the only way is to edit the snap.conf file directly with a text editor.


#10

Another way to approach the java heap space error: flush memory. Save products, save the session, close SNAP, open SNAP, select File>Reopen Product. Select only the product needed. Alternatively, select File>Session>Open Session, select the session. Some operations, viewing images seems to hold memory, this removes that load.

Also look at creating virtual memory. This is emulated memory on a drive. I have not tested utility of virtual memory by SNAP.


#11

If by that you mean enabling “swapping”, I strongly advise not to use it. Your computer will become extremely slow or even not responding at all.


#12

My understanding of “swapping” or “disk swapping” is when a multitasking operating system takes an image of memory used for program execution, writes it to a disk, when the program is idle, freeing memory for active program execution. That’s memory management in multitasking environment. Noticeably slow on old hardware, much faster with new, faster bus, faster drive speed.

Virtual memory is similar, where much slower disk is used to increase the amount of virtual memory. However, I have seen virtual memory, showing as “enabled” by default, on a new machine. Still, I have not tested use of virtual memory.

I did watch memory usage by SNAP and solved the java heap space error by restarting SNAP.


#13
  • Check the system disk usage. If the disk is full, clean up as much as possible the disk, the trash and the SNAP cache.

(check your disk usage with df using command line), for SNAP cash ( you must go to this direction to check this: var/cache to remove tmp files, in the case of your processing, because you use Sentinel1 processing SNAP doesn’t make temporary files in cache folder It makes just for Sentinel2 processing.)

-increase the minimum and maximum java heap size. To do this, modify snap.conf in /usr/local/snapx/etc. Then increase parameters Xms and XMx(respectively minimum and maximum memory size), without exceeding the allocated RAM of the VM.( you can check your memory with free command in command line). Then open with emacs snap.conf file ( you must use it with sudo otherwise it’s just read only and you can’t modify it). you can also increase parameters Xms and Xmx of gpt.vmoptions file in /usr/local/snapx/bin.


#14

I found that setting the the VM parameter -Xmx , maximum size, to be a critical setting. A value about half the size of total memory has been working. The parameter can be changed in Options>Performance tab, or be edited the configuration file with a text editor. -Xms , minimum size, may be small (256M). Also useful to set the cache size, the medium value of the benchmark tests seems to work better than the smaller, default, value.


#15

I ran into java heap space when performing all swath split in SNAP, and I have gone through the contributions on the subject matter but non worked in my favour yet. Can someone simplify the process of resolving heap space error? What do i need to change on the dialogue box (tool -> option https://forum.step.esa.int/uploads/default/original/2X/0/024ae75804f4c48f72e97245c1f5bf3b40b2e0c1.png). I have also increased the SNAP values but no solution yet. I also tried to remove the tmp files but permission denied.

Please I need some help.


#16

Hi Samira.h , I tried to remove the tmp file as you suggested but it did not work for me. I got permission denied. Please help.


#17

Increasing the available memory only has an effect if your PC has this amount of RAM. It only defines how much of your RAM can be used by Java (SNAP in this case).
For example, if you have 8 GB of RAM, increasing the max value to 16 GB has no effect.
m in this context stands or megabytes and the values should ideally be a multiple of 512 (explanation)
You can more flexibly also use G instead: 2G are 2048m, 4G are 4096m, and so on…

So if you have 16 GB of RAM, you can set it to -Xmx12G
If you only have 8 GB of RAM, you can set it to -Xmx6G


Java Heap Space Error