Java Heap Space Error

[JBrownett]

I keep getting a Java Heap Space error when trying to run the following process:

Its using a 10m External DEM, and Refined Lee Speckle Filter but otherwise is just running using Default Settings and I cannot understand whats causing this issue. From time to time I also get a GC Overhead Limit exceeded message. The PC is an Intel I7, with 16gb ram running Windows 10 64 bit, so it should handle this fine.

Any help on solving this issue please?

[ABraun]

from another topic:

Problems obtaining an interferogram from two product sets

Monster-graphs are usually a lot slower than cutting the work into smaller pieces.

[JBrownett]

Thanks, Is there a way to deal with this, other than cutting down larger graphs! Would it be more capable through Command Line for example? I ask because I am only trying to run Terrain Filter and Terrain Correction now and I am still getting Java Heap Space issues!

[marpet]

Do you let the process run from the command line or directly within SNAP Desktop?
In Desktop you can try to increase the cache.
Go to Tools\Options and switch to the Performance tab. Set the Cache size to a higher value. e.g. 4096.
You can also try to use the file cache. Therefor you need to change the option on the S1TBX tab.
If you use gpt on the command line you can specify the -c parameter. Probably you also need to set the -q option. Maybe to something like 120. See

Batch processing of biophysical retirevals using multiple CPUs

I was able to reproduce the problem you have described. It seems there is an issue when using operators in a graph which implement a certain method (computeTileStack()). In this case the parallelisation isn't working correct. When using the operator directly, without a graph, the CPU usage is 100%. For me it dropped to 18% when used in a graph. After specifying the -q option with a value of 120, I got the 100% again. It increases the number of allowed threads to 120. Maybe you have to play a bit…

[JBrownett]

I have been running the process directly from SNAP Desktop, but am also now trying to see if I can get it to run from Command Line, following your guidance on the -q option.

The graph currently has an error, but I cannot see how to resolve it. As far as I can tell I have the code in correctly, can you see an issue? The error I am getting is that it cannot write product file: ‘C:\Program Files\snap\bin$target1’

Command:
gpt “E:\MODELS\SNAP\S1_PROCESSING\S1_PROCESSING_BATCH
.xml” -Pinput1=“E:_PROJECTS\RAW\WINTER\S1A_IW_GRDH_1SDV_20160108T063109_20160108T063134_009399_00D9C6_8DAF.zip” -t target1=“E:_PROJECTS\PROCESSED\OUTPUT_2016_01_08.tif”

Graph:
S1_PROCESSING_BATCH (5.7 KB)

[marpet]

The problem is the usage of the -t option.
You can specify -t and -f (the target format name) as follows:

... -t  "E:_PROJECTS\PROCESSED\OUTPUT_2016_01_08.tif" -f GeoTIFF-BigTIFF

In this case you need to remove the “Write” node from your graph.

Or you specify target1 as parameter:

... -Ptarget1=="E:_PROJECTS\PROCESSED\OUTPUT_2016_01_08.tif"

[JBrownett]

Thanks, the process is trying to run now. Still having issues with it though. Firstly, I get a message called Severe:

INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Loading external tools…
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Scanning for external tools adapters: C:\Program Files\snap\smos
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Scanning for external tools adapters: C:\Program Files\snap\s3tbx
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Scanning for external tools adapters: C:\Program Files\snap\rstb
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Scanning for external tools adapters: C:\Program Files\snap\s2tbx
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Scanning for external tools adapters: C:\Users\GIS-User.snap\auxdata\tool-adapters
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Scanning for external tools adapters: C:\Program Files\snap\java
SEVERE: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Failed scan for Tools descriptors: I/O problem: C:\Program Files\snap\java

Is this something to worry about?

Secondly, the process is still stopping, with the message GC overhead limit exceeded.

GC overhead limit exceeded
GC overhead limit exceeded
GC overhead limit exceeded
GC overhead limit exceeded
GC overhead limit exceeded
GC overhead limit exceeded
java.lang.NullPointerException
java.lang.NullPointerException
java.lang.NullPointerException
java.lang.NullPointerException
java.lang.NullPointerException
java.lang.NullPointerException
java.lang.NullPointerException
java.lang.NullPointerException
org.esa.snap.core.gpf.OperatorException: GC overhead limit exceeded
at org.esa.snap.core.gpf.graph.GraphProcessor$GPFImagingListener.errorOccurred(GraphProcessor.java:373)
at com.sun.media.jai.util.SunTileScheduler.sendExceptionToListener(SunTileScheduler.java:1646)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:921)
at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
at org.esa.s1tbx.sar.gpf.geometric.TerrainFlatteningOp.outputNormalizedImageGamma0(TerrainFlatteningOp.java:840)
at org.esa.s1tbx.sar.gpf.geometric.TerrainFlatteningOp.computeTileStack(TerrainFlatteningOp.java:484)
at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeRect(OperatorImageTileStack.java:116)
GC overhead limit exceeded at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeTile(OperatorImageTileStack.java:85)

    at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)Cannot construct DataBuffer.

    at javax.media.jai.OpImage.getTile(OpImage.java:1129)

Cannot construct DataBuffer.
at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
GC overhead limit exceeded at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)

    at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)GC overhead limit exceeded

    at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)

GC overhead limit exceeded
at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.computeTileStack(RangeDopplerGeocodingOp.java:935)Cannot construct DataBuffer.

    at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeRect(OperatorImageTileStack.java:116)
    at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeTile(OperatorImageTileStack.java:85)Cannot construct DataBuffer.
    at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)

Cannot construct DataBuffer. at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:73)
at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at com.sun.media.jai.util.RequestJob.compute(SunTileScheduler.java:247)
at com.sun.media.jai.util.WorkerThread.run(SunTileScheduler.java:468)

Cannot construct DataBuffer.
Caused by: org.esa.snap.core.gpf.OperatorException: GC overhead limit exceededGC overhead limit exceeded
at org.esa.snap.engine_utilities.gpf.OperatorUtils.catchOperatorException(OperatorUtils.java:421)
at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilterOp.computeTile(SpeckleFilterOp.java:301)

    at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
    at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)GC overhead limit exceeded

    at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
    ... 33 more

Error: GC overhead limit exceeded
GC overhead limit exceededorg.esa.snap.core.gpf.OperatorException: GC overhead limit exceeded
at org.esa.s1tbx.calibration.gpf.Sentinel1RemoveThermalNoiseOp.computeTile(Sentinel1RemoveThermalNoiseOp.java:540)
at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
at org.esa.s1tbx.calibration.gpf.calibrators.Sentinel1Calibrator.computeTile(Sentinel1Calibrator.java:529)
at org.esa.s1tbx.calibration.gpf.CalibrationOp.computeTile(CalibrationOp.java:188)
at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)

GC overhead limit exceeded at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilters.RefinedLee.performFiltering(RefinedLee.java:111)
at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilters.RefinedLee.computeTile(RefinedLee.java:77)

GC overhead limit exceeded at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilterOp.computeTile(SpeckleFilterOp.java:299)
GC overhead limit exceeded
at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)

    at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)

GC overhead limit exceeded at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)

GC overhead limit exceeded at javax.media.jai.OpImage.getTile(OpImage.java:1129)
GC overhead limit exceeded

    at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)GC overhead limit exceeded

    at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)

GC overhead limit exceeded at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)

    at org.esa.s1tbx.sar.gpf.geometric.TerrainFlatteningOp.outputNormalizedImageGamma0(TerrainFlatteningOp.java:840)
    at org.esa.s1tbx.sar.gpf.geometric.TerrainFlatteningOp.computeTileStack(TerrainFlatteningOp.java:484)
    at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeRect(OperatorImageTileStack.java:116)GC overhead limit exceeded

GC overhead limit exceeded
at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeTile(OperatorImageTileStack.java:85)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
GC overhead limit exceeded at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)

    at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)GC overhead limit exceeded

    at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
    at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
    at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
    at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
    at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.computeTileStack(RangeDopplerGeocodingOp.java:935)
    at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeRect(OperatorImageTileStack.java:116)
    at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeTile(OperatorImageTileStack.java:85)
    at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
    at javax.media.jai.OpImage.getTile(OpImage.java:1129)
    at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
    at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
    at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
    at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
    at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:73)
    at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
    at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
    at javax.media.jai.OpImage.getTile(OpImage.java:1129)
    at com.sun.media.jai.util.RequestJob.compute(SunTileScheduler.java:247)
    at com.sun.media.jai.util.WorkerThread.run(SunTileScheduler.java:468)

Any ideas what to do to resolve this?

Thanks.

[marpet]

The first INFO and SEVERE messages are unnecessary and also wrong messages. Please ignore them. WE have it already corrected in our source code.
Maybe this topic is helpful:

GPT Sentinel 1 TOPSAR coherence graph: GC overhead limit exceeded

Found a solution at http://forum.step.esa.int/t/gpt-performance/460 This is what I did: snap-conf-optimiser compute the best system options (the top compute button) and copy the VM parameters from the top line gedit $SNAP_HOME_DIRECTORY/gpt.vmoptions paste each parameter from the top line in snap-conf-optimiser to a new line in gpt.vmoptions Hope that helps others with same issue. James

And maybe these:

• Terrain-Correction: GC overhead limit exceeded
• Gpt performance

[JBrownett]

Thanks.

I have had a read and am testing again now, hopefully the solutions in those topics will solve it!

[khaldi]

hello, i am having the same error " Java heap space" ! i m no a pro in this actually it’s my first time using the snap so please can u explain this issue more to me and how can i solve it.
i am using PC laptop with 8GO ram, windos 8 64 bit.
PS; i’ve use the snap successfully to get coregistration last time but an error in the interferogram formation accured so i had to uninstall it and install it all over again and now i’am stuck even befor doing the coregistration.

[marpet]

You can try to slightly increase the value for the heap space. To do so go to etc/snap.conf. But I think it has already a good value.

It might be better to try to lower the tile size and/or lower the parallelism of the processing.
In the etc/snap.properties change the following
snap.jai.defaultTileSize = 512 --> snap.jai.defaultTileSize = 200
#snap.parallelism = 1 --> snap.parallelism = 2

With the default setting all available cores will be used. Setting it to a lower value will reduce the memory used at the same time.

[khaldi]

ok thank u for your help

[janardanroy]

i’m facing the same problem with SNAP-6.0. What should i do now?

[ABraun]

read the suggestion given in this topic

[shubhangsk]

Greetings ABraun,

I’m trying to coregister the images using DEM. I’m also facing the java.lang.NullPointerException error. I’m working on a workstation with 256 GB RAM and 3 TB free Hard Disk space. I’ve followed all you previous suggestions (increasing cache to 4096 and to check the box for ‘Use FileCache in readers to conserve memory’ but none of them worked.
Please guide for any other solutions that could be used.

Best Regards
Shubhang

[marpet]

A NullPointerException indicates usually some other error, only in some cases, it is caused by a memory issue.
Can you attach the log file please? You can find it by selecting from the menu Help / Show Log Directory. Make sure to reproduce the error first and then attach the messages.log file.

[shubhangsk]

messages.log (67.3 KB)

Greetings marpret,

Please find the attached messages.log file. This file is created now with the same error occurring again.

Regards

[khanwaleed247]

org.esa.snap.core.gpf.OperatorException: Java heap space
at org.esa.s1tbx.calibration.gpf.Sentinel1RemoveThermalNoiseOp.computeTile(Sentinel1RemoveThermalNoiseOp.java:635)
at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
Caused: org.esa.snap.core.gpf.OperatorException: Java heap space
at org.esa.snap.core.gpf.internal.OperatorExecutor$GPFImagingListener.errorOccurred(OperatorExecutor.java:376)
at com.sun.media.jai.util.SunTileScheduler.sendExceptionToListener(SunTileScheduler.java:1646)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:921)
at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:73)
at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at com.sun.media.jai.util.RequestJob.compute(SunTileScheduler.java:247)
[catch] at com.sun.media.jai.util.WorkerThread.run(SunTileScheduler.java:468)

[ABraun]

Please don’t just post error messages. Tell us about your system (especially RAM) and your data processing.
Many answers and solutions have been proposed for this error im this form. Which ones did you try so far?

[khanwaleed247]

Hope you are fine and doing well.
Thank you very much for your reply, here are the solutions I tried after searching the FAQs.

1. I upgraded my system RAM from 8 Gb to 16 Gb.
2. I even tried increasing the size of xmx parameter to 32Gb in SNAP interface.

After applying the above settings for my 5th gen laptop, Thermal noise filtering failed with Java Heap Space.