Terrain-Correction: GC overhead limit exceeded

Microwave Toolbox
#1

Hello,

I’m coming with an external Terrain correction problem which seems to propagate in the following graph with GPT. I’m testing simple graph before going further.

Pasted image1035×149 14.7 KB

Command : C:/Program Files/snap/bin/gpt C:/…/SLC_to_GRD_orbit_Cal_Thermal_Multi_TC_MNSLidar.xml -e -c 93350M -q 8 -PinFile=C:/…/S1A_IW_SLC__1SDV_20150821T055817_20150821T055844_007357_00A1B9_4D44.zip -PoutFile=C:/…/S1A_IW_SLC__1SDV_20150821T055817_20150821T055844_007357_00A1B9_4D4_TC

When processing data with SRTM 1s (auto download), GPT seems to do its job. When going with an external DTM (1m resolution), I get the error pasted bellow.

My config (on server):SNAP 2.0.1 + s1tbx 2.0.2 ( 128 Go RAM, no HDD memory limit ).

The GC overhead limit indicate a memory problem but GPT works with more than 50 Go RAM free besides it. Furthermore, it takes hours to go for the processing (before failure) using such DTM at 1m (while it takes 70 minutes with SRTM). Any option to reduce computation time with high resolution?

Thank you for any answer !

Full error message :

TOPSAR-Deburst: Cannot construct DataBuffer.
Multilook: java.lang.NullPointerException
Multilook: java.lang.NullPointerException
TOPSAR-Deburst: java.lang.NullPointerException
Multilook: java.lang.NullPointerException
Multilook: java.lang.NullPointerException
Multilook: java.lang.NullPointerException
TOPSAR-Deburst: GC overhead limit exceeded
Multilook: java.lang.NullPointerException
Multilook: java.lang.NullPointerException
Multilook: java.lang.NullPointerException
Multilook: Cannot construct DataBuffer.
TOPSAR-Deburst: Cannot construct DataBuffer.
Multilook: Cannot construct DataBuffer.
Multilook: java.lang.NullPointerException
Terrain-Correction: GC overhead limit exceeded
Terrain-Correction: Terrain-Correction: GC overhead limit exceeded
org.esa.snap.core.gpf.OperatorException: TOPSAR-Deburst: Cannot construct DataBuffer.

  • at org.esa.snap.core.gpf.graph.GraphProcessor$GPFImagingListener.errorOccurred(GraphProcessor.java:373)*
  • at com.sun.media.jai.util.SunTileScheduler.sendExceptionToListener(SunTileScheduler.java:1646)*
  • at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:921)*
  • at javax.media.jai.OpImage.getTile(OpImage.java:1129)*
  • at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)*
  • at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)*
  • at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)*
  • at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:420)*
  • at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:406)*
  • at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:442)*
  • at org.esa.s1tbx.sar.gpf.MultilookOp.computeTile(MultilookOp.java:180)*
  • at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)*
  • at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)*
  • at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)*
  • at javax.media.jai.OpImage.getTile(OpImage.java:1129)*
  • at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)*
  • at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)*
  • at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)*
  • at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:420)*
  • at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:406)*
  • at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:442)*
  • at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.computeTileStack(RangeDopplerGeocodingOp.java:911)*
  • at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeRect(OperatorImageTileStack.java:116)*
  • at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeTile(OperatorImageTileStack.java:85)*
  • at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)*
  • at javax.media.jai.OpImage.getTile(OpImage.java:1129)*
  • at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)*
  • at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)*
  • at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:420)*
  • at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:406)*
  • at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:73)*
  • at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)*
  • at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)*
  • at javax.media.jai.OpImage.getTile(OpImage.java:1129)*
  • at com.sun.media.jai.util.RequestJob.compute(SunTileScheduler.java:247)*
  • at com.sun.media.jai.util.WorkerThread.run(SunTileScheduler.java:468)*
    Caused by: org.esa.snap.core.gpf.OperatorException: TOPSAR-Deburst: Cannot construct DataBuffer.
  • at org.esa.snap.engine_utilities.gpf.OperatorUtils.catchOperatorException(OperatorUtils.java:386)*
  • at org.esa.s1tbx.sentinel1.gpf.TOPSARDeburstOp.computeTileStack(TOPSARDeburstOp.java:798)*
  • at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeRect(OperatorImageTileStack.java:116)*
  • at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeTile(OperatorImageTileStack.java:85)*
  • at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)*
  • … 33 more*

Error: TOPSAR-Deburst: Cannot construct DataBuffer.
TOPSAR-Deburst: GC overhead limit exceeded
Multilook: TOPSAR-Deburst: GC overhead limit exceeded

Java Heap Space Error
gc overhead limit exceeded
S1 TOPS Coregistration: GC overhead limit exceeded
Problem with interferometry
Sentinel-1 data preprocessing
#2

You are out of RAM and/or disk. If you are forcing the output to be in 1m resolution that could easily happen. Also, the larger the graphs are the more memory is consumed.

edit: There is no benefit in using a DEM in higher resolution than the satellite data. You should select the output pixel-size accordingly.

2 Likes
#3

Thank you for the answer. I didn’t noticed RAM or HDD exceeding during that processing. Furthermore, the -c parameter should be limiting overflowed memory, right? If not, ,what should I do to limit the memory in command line?

There is no benefit in using a DEM in higher resolution than the satellite data. You should select the output pixel-size accordingly.
Output pixel in terrain correction is of 15m*15m (specified in TC parameters). Is this what you meant? Therefore, it should also limits memory problems.

#4

S-1 GRD has a pixel-spacing of 10m so you could try that. If you still run into problems you could first reduce the resolution of the DEM.

#5

You could try to break it down into two graphs. After mulitlook put the GRD-post processing which just renames the product and changes its product type and write it, then terrain correct as a separate step which is probably the heaviest.

Also, update georeference is only intended to update the geocoding in the product similar to terrain correction but keeping the data in SAR geometry and instead creating a pixel geocoding of where the lats/lons would be if the product was terrain corrected. This is only needed if you want to overlay shape files in SAR geometry and you want them to appear on the right place on the ground.

If you are going to terrain correct anyway in your graph then it’s totally useless to do update georefernce and will really slow things down.

-c sets the cache size but you still need to make sure the Java JVM has enough heap space

1 Like
S1 TOPS Coregistration "Cannot Create Data Buffer"
#6

Dividing the graph was the solution.

May I suggest to add the memory management while processing a graph in SNAP development.

Have a nice day.

#7

Hi @Guillaume,
I wonder that how can you find 1m resolution DTM ?? I am using SRTM 1 arc second DTM which has 30 m resolution. Is it your own data that obtain from field works??

regards
Fikret

#8

In my case just clicking the Compute tab in
“Tools\Options\Performance” helped. Even though it reduced the Cache Size. That’s quite counterintuitive.