Java heap space / Data Buffer error when using gpt tool (terrain correction)

wolf · October 20, 2015, 9:23pm

I have created a graph for the processing of one S1 image (calibration, terrain correction, conversion to db, speckle filter, write to tif) that worked very well in the desktop version of SNAP (8 minutes processing time). However, when I run the xml- file using gpt in the cmd line, I get problems with Data Buffer or Java Heap Space. The process is stopped and the resulting tif only shows image information in the top area. I am running a Win7-64 bit machine with 8 GB RAM.

Here is the error message:

C:\Users\Wolfgang>gpt D:\zmon\Code\fullGraphV1.xml -PinFile="D:\zmon\Code\input\
S1A_S4_GRDH_1SDH_20141111T171431_20141111T171455_003237_003BC9_7711.zip" -PoutFi
le="D:\zmon\Code\output\S1A_S4_GRDH_1SDH_20141111T171431_20141111T171455_003237_
003BC9_7711_TC1.tif"
Terrain-Correction: java.lang.NullPointerException
Terrain-Correction: Terrain-Correction: java.lang.NullPointerException
LinearTodB: java.lang.NullPointerException
Speckle-Filter: java.lang.NullPointerException
Speckle-Filter: Waiting thread received a null tile.
Speckle-Filter: Waiting thread received a null tile.
org.esa.snap.core.gpf.OperatorException: Cannot construct DataBuffer.
        at org.esa.snap.core.gpf.graph.GraphProcessor$GPFImagingListener.errorOc
curred(GraphProcessor.java:373)
        at com.sun.media.jai.util.SunTileScheduler.sendExceptionToListener(SunTi
leScheduler.java:1646)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler
.java:921)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(Operator
Context.java:420)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(Operator
Context.java:406)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:440)
        at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.getPixelValue
(RangeDopplerGeocodingOp.java:1112)
        at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.computeTileSt
ack(RangeDopplerGeocodingOp.java:1025)
        at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeRect(Ope
ratorImageTileStack.java:116)
        at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeTile(Ope
ratorImageTileStack.java:85)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler
.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(Operator
Context.java:420)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(Operator
Context.java:406)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:440)
        at org.esa.s1tbx.utilities.gpf.LinearTodBOp.computeTile(LinearTodBOp.jav
a:114)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImag
e.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:
137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler
.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(Operator
Context.java:420)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(Operator
Context.java:406)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:440)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilterOp.computeTile(SpeckleFi
lterOp.java:232)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImag
e.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:
137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler
.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(Operator
Context.java:420)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(Operator
Context.java:406)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImag
e.java:73)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:
137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler
.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at com.sun.media.jai.util.RequestJob.compute(SunTileScheduler.java:247)
        at com.sun.media.jai.util.WorkerThread.run(SunTileScheduler.java:468)
Caused by: java.lang.RuntimeException: Cannot construct DataBuffer.
        at com.sun.media.jai.util.DataBufferUtils.constructDataBuffer(DataBuffer
Utils.java:132)
        at com.sun.media.jai.util.DataBufferUtils.createDataBufferFloat(DataBuff
erUtils.java:214)
        at javax.media.jai.ComponentSampleModelJAI.createDataBuffer(ComponentSam
pleModelJAI.java:271)
        at javax.media.jai.RasterFactory.createWritableRaster(RasterFactory.java
:691)
        at javax.media.jai.PlanarImage.createWritableRaster(PlanarImage.java:198
2)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:
130)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler
.java:904)
        ... 44 more

Error: Cannot construct DataBuffer.

Here is the contents of my xml-file:

<graph id="Graph">
  <version>1.0</version>
  <node id="Read">
    <operator>Read</operator>
    <sources/>
    <parameters class="com.bc.ceres.binding.dom.XppDomElement">
      <file>${inFile}</file>
      <formatName>SENTINEL-1</formatName>
    </parameters>
  </node>
  <node id="Calibration">
    <operator>Calibration</operator>
    <sources>
      <sourceProduct refid="Read"/>
    </sources>
    <parameters class="com.bc.ceres.binding.dom.XppDomElement">
      <sourceBands/>
      <auxFile>Product Auxiliary File</auxFile>
      <externalAuxFile/>
      <outputImageInComplex>false</outputImageInComplex>
      <outputImageScaleInDb>false</outputImageScaleInDb>
      <createGammaBand>false</createGammaBand>
      <createBetaBand>false</createBetaBand>
      <selectedPolarisations>HH</selectedPolarisations>
      <outputSigmaBand>true</outputSigmaBand>
      <outputGammaBand>false</outputGammaBand>
      <outputBetaBand>false</outputBetaBand>
      <outputDNBand>false</outputDNBand>
    </parameters>
  </node>
  <node id="Terrain-Correction">
    <operator>Terrain-Correction</operator>
    <sources>
      <sourceProduct refid="Calibration"/>
    </sources>
    <parameters class="com.bc.ceres.binding.dom.XppDomElement">
      <sourceBands>Sigma0_HH</sourceBands>
      <demName>SRTM 3Sec</demName>
      <externalDEMFile/>
      <externalDEMNoDataValue>0.0</externalDEMNoDataValue>
      <demResamplingMethod>BILINEAR_INTERPOLATION</demResamplingMethod>
      <imgResamplingMethod>BILINEAR_INTERPOLATION</imgResamplingMethod>
      <pixelSpacingInMeter>10.0</pixelSpacingInMeter>
      <pixelSpacingInDegree>8.983152841195215E-5</pixelSpacingInDegree>
      <mapProjection>GEOGCS[&quot;WGS84(DD)&quot;, &#xd;
  DATUM[&quot;WGS84&quot;, &#xd;
    SPHEROID[&quot;WGS84&quot;, 6378137.0, 298.257223563]], &#xd;
  PRIMEM[&quot;Greenwich&quot;, 0.0], &#xd;
  UNIT[&quot;degree&quot;, 0.017453292519943295], &#xd;
  AXIS[&quot;Geodetic longitude&quot;, EAST], &#xd;
  AXIS[&quot;Geodetic latitude&quot;, NORTH]]</mapProjection>
      <nodataValueAtSea>true</nodataValueAtSea>
      <saveDEM>false</saveDEM>
      <saveLatLon>false</saveLatLon>
      <saveIncidenceAngleFromEllipsoid>false</saveIncidenceAngleFromEllipsoid>
      <saveLocalIncidenceAngle>false</saveLocalIncidenceAngle>
      <saveProjectedLocalIncidenceAngle>false</saveProjectedLocalIncidenceAngle>
      <saveSelectedSourceBand>true</saveSelectedSourceBand>
      <applyRadiometricNormalization>false</applyRadiometricNormalization>
      <saveSigmaNought>false</saveSigmaNought>
      <saveGammaNought>false</saveGammaNought>
      <saveBetaNought>false</saveBetaNought>
      <incidenceAngleForSigma0>Use projected local incidence angle from DEM</incidenceAngleForSigma0>
      <incidenceAngleForGamma0>Use projected local incidence angle from DEM</incidenceAngleForGamma0>
      <auxFile>Latest Auxiliary File</auxFile>
      <externalAuxFile/>
    </parameters>
  </node>
  <node id="LinearTodB">
    <operator>LinearTodB</operator>
    <sources>
      <sourceProduct refid="Terrain-Correction"/>
    </sources>
    <parameters class="com.bc.ceres.binding.dom.XppDomElement">
      <sourceBands>Sigma0_HH</sourceBands>
    </parameters>
  </node>
  <node id="Speckle-Filter">
    <operator>Speckle-Filter</operator>
    <sources>
      <sourceProduct refid="LinearTodB"/>
    </sources>
    <parameters class="com.bc.ceres.binding.dom.XppDomElement">
      <sourceBands>Sigma0_HH_db</sourceBands>
      <filter>Median</filter>
      <filterSizeX>3</filterSizeX>
      <filterSizeY>3</filterSizeY>
      <dampingFactor>2</dampingFactor>
      <edgeThreshold>5000.0</edgeThreshold>
      <estimateENL>true</estimateENL>
      <enl>1.0</enl>
    </parameters>
  </node>
  <node id="Write">
    <operator>Write</operator>
    <sources>
      <sourceProduct refid="Speckle-Filter"/>
    </sources>
    <parameters class="com.bc.ceres.binding.dom.XppDomElement">
      <file>${outFile}</file>
      <formatName>GeoTIFF</formatName>
    </parameters>
  </node>
  <applicationData id="Presentation">
    <Description/>
    <node id="Speckle-Filter">
      <displayPosition x="328.0" y="185.0"/>
    </node>
    <node id="LinearTodB">
      <displayPosition x="216.0" y="138.0"/>
    </node>
    <node id="Terrain-Correction">
      <displayPosition x="115.0" y="92.0"/>
    </node>
    <node id="Calibration">
      <displayPosition x="66.0" y="58.0"/>
    </node>
    <node id="Read">
            <displayPosition x="17.0" y="21.0"/>
    </node>
    <node id="Write">
            <displayPosition x="429.0" y="236.0"/>
    </node>
  </applicationData>
</graph>

snap.conf contains the following: -J-Xms256M -J-Xmx5G

But that should be fine?

javier_unib · May 29, 2016, 7:14am

I am having the same problem, but with the desktop version.
My processing chain is split, deburst, polarimetric matrix, terrain correction and write to tiff.

This workflow used to work always but now I have noticed that now I can only make it work once. The following times I run it (even with the same image) I get the java heap or the cannot construct data buffer errors. Might it be that temporal files are still using resources in the computer? I haven’t manage to remove them. The approach I am following for the moment is switching computers, and I am running out of them.

ASFStoner · September 22, 2016, 6:09pm

I’m seeing this too using oil spill detection but it won’t run in Desktop version nor in gpt. Other graphs run, but this one immediately crashes with null tile: “Waiting thread received a null tile.”

kadammanali987 · January 9, 2019, 6:43am

I am also having same issue.

ABraun · January 9, 2019, 7:47am

8 GB RAM is mostly not enough for SAR processing

mengdahl · January 9, 2019, 10:06am

It should be possible but it requires tweaking Java VM parameters, use of FileCache in reading and working with short and simple graphs one after the other. We are looking into the issue.

kadammanali987 · January 28, 2019, 8:29am

Thanks for the excellent answer.