Error: Cannot construct DataBuffer

snap
1

Hi,

I’m getting both the cannot construct DataBuffer and sometimes the data heap space error.

I have tried increasing both the min and max memory in VM parameters but the error still persists.

Here is the log for the Data Buffer error:

java.lang.RuntimeException: Cannot construct DataBuffer.
at com.sun.media.jai.util.DataBufferUtils.constructDataBuffer(DataBufferUtils.java:132)
at com.sun.media.jai.util.DataBufferUtils.createDataBufferFloat(DataBufferUtils.java:214)
at javax.media.jai.ComponentSampleModelJAI.createDataBuffer(ComponentSampleModelJAI.java:271)
at javax.media.jai.RasterFactory.createWritableRaster(RasterFactory.java:691)
at javax.media.jai.PlanarImage.createWritableRaster(PlanarImage.java:1982)
at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:130)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
Caused: org.esa.snap.framework.gpf.OperatorException: Cannot construct DataBuffer.
at org.esa.snap.framework.gpf.internal.OperatorExecutor$GPFImagingListener.errorOccurred(OperatorExecutor.java:375)
at com.sun.media.jai.util.SunTileScheduler.sendExceptionToListener(SunTileScheduler.java:1646)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:921)
at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
at org.esa.snap.framework.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:421)
at org.esa.snap.framework.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
at org.esa.snap.framework.gpf.internal.OperatorImage.computeRect(OperatorImage.java:73)
at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at com.sun.media.jai.util.RequestJob.compute(SunTileScheduler.java:247)
[catch] at com.sun.media.jai.util.WorkerThread.run(SunTileScheduler.java:468)

1 Like
2

Hi,
to which memory value have you set the VM parameter? What type of data are you using and which operator do you run?

3

I let SNAP compute the optimal memory usage. I did try setting it myself but it made no difference. FYI, the parameters that were optimised were -Xmx2957m and -Xms2048m. Optimisation of the parameters did not help .

I tried running the speckle filter and also the calibration operation using default settings. Both failed yielding either the DataBuffer or the heap space error which points to a memory issue.

I was trying to process the Sentinel-1 IW data which I downloaded off the ESA hub.

4

How much RAM do you have? 16GB is the recommended amount and 8GB seems to work quite well but with 4GB people seem to have problems.

2 Likes
5

8GB, my resource monitor indicates not even half were fully utilised. Anyway, the imagery is only about 800MB…

6

The more memory you have the better. But it should be enough memory. Actually most of the operations in SNAP work also with 4GB. Maybe the operations (speckle filter, calibration) you run are very memory hungry. I haven’t used them. But it might be also worth to see if the implementation could be improved.

7

I think you could try increasing the -Xmx parameter to for example 5GB and see if that helps.

1 Like
8

I have already increased it to 6GB and the error persists. I didn’t think a simple filter operation would take more than 4GB of memory? Keen to hear more from the devs or anyone else if they are having the same problem as me.

9

I keep having these issues as well and it is exceedingly frustrating. It seems that I can’t do anything with the software. I have found several tutorials yet can’t get more than the first step into them as I can’t simply deburst the data for the starting point.

1 Like
10

Hello,

I agree that it’s frustrating to have powerfull software that Unfortunately works well only on huge computer.

If it’s one develpoment target I think that it would be very nice that memory optimisation of snap would be in the roadmap in order that tools work at least on standard computer with 4go ram even if it’s slow.
Indeed many people can’t buy software at 1000 or 5000€ to process sar data, and in my point of view snap is THE free sar software to use.

Thanks for support and frequent update.

11

In order to better understand why you keep getting this error, we need to exactly reproduce your case. Could you please identify the data product, the operators and settings so we can start investigating.

Memory management has always been an important design aspect in BEAM, followed by NEST, and now SNAP and its improvement is definitely on the road map. However, high memory usage may also result from specific operator implementations (whose algorithms require that, or because they are not carefully implemented) and not directly from the general SNAP memory management.

12

Thanks to more detailed information about memory management that help to better understand development constraint.

13

Hi Forman,

Here are all the available information I can provide:

Data Product : Sentiel 1 IW GRDH (S1A_IW_GRDH_1SSV_20150710)
Operators : Speckle Filtering (Single Product->Save as BEAM-DIMAP, Source bands : Intensity and Amp, Filter : Refined Lee at 5000.) , Calibration and probably others but I haven’t tried them

Performance Parameters :
VM param:
-Xmx2417m
-Xms2048m
-XX:+AggressiveOpts
-Xverify:none
-Dnetbeans.mainclass=org.esa.snap.main.Main
-Dsun.java2d.noddraw=true
-Dsun.awt.nopixfmt=true
-Dsun.java2d.dpiaware=false

Tile sz :512px
Cache size : 1024Mb
No of threads: 4

System Spec (DXDIAG )

System Information

Time of this report: 9/15/2015, 16:35:56
Machine name: AU-D13272
Operating System: Windows 7 Enterprise 64-bit (6.1, Build 7601) Service Pack 1 (7601.win7sp1_ldr.150715-0600)
Language: English (Regional Setting: English)
System Manufacturer: Dell Inc.
System Model: OptiPlex 9010
BIOS: BIOS Date: 09/19/12 10:43:37 Ver: A08.00
Processor: Intel® Core™ i5-3470 CPU @ 3.20GHz (4 CPUs), ~3.2GHz
Memory: 8192MB RAM
Available OS Memory: 8146MB RAM
Page File: 6557MB used, 9733MB available
Windows Dir: C:\Windows
DirectX Version: DirectX 11
DX Setup Parameters: Not found
User DPI Setting: Using System DPI
System DPI Setting: 96 DPI (100 percent)
DWM DPI Scaling: Disabled
DxDiag Version: 6.01.7601.17514 64bit Unicode

I think the SNAP has its own JRE installation in the SNAP directory so JRE version shouldn’t be an issue?

Hopefully you can replicate the problem.

Thanks

14

Great, thanks for the exhaustive info!
– Norman

15

Meanwhile could you please try the following. In your etc/snap.properties specify the following settings and run again:

snap.gpf.disableTileCache = true
snap.gpf.useFileTileCache = false

Then please also try to run again with

snap.gpf.disableTileCache = false
snap.gpf.useFileTileCache = true
16

Thanks for the follow up forman!

I’m still running into the data buffer error again :smile:java.lang.RuntimeException: Cannot construct DataBuffer.
at com.sun.media.jai.util.DataBufferUtils.constructDataBuffer(DataBufferUtils.java:132)
at com.sun.media.jai.util.DataBufferUtils.createDataBufferFloat(DataBufferUtils.java:214)
at javax.media.jai.ComponentSampleModelJAI.createDataBuffer(ComponentSampleModelJAI.java:271)
at javax.media.jai.RasterFactory.createWritableRaster(RasterFactory.java:691)
at javax.media.jai.PlanarImage.createWritableRaster(PlanarImage.java:1982)
at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:130)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
Caused: org.esa.snap.framework.gpf.OperatorException: Cannot construct DataBuffer.
at org.esa.snap.framework.gpf.internal.OperatorExecutor$GPFImagingListener.errorOccurred(OperatorExecutor.java:375)
at com.sun.media.jai.util.SunTileScheduler.sendExceptionToListener(SunTileScheduler.java:1646)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:921)
at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
at org.esa.snap.framework.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:421)
at org.esa.snap.framework.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
at org.esa.snap.framework.gpf.internal.OperatorImage.computeRect(OperatorImage.java:73)
at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at com.sun.media.jai.util.RequestJob.compute(SunTileScheduler.java:247)
[catch] at com.sun.media.jai.util.WorkerThread.run(SunTileScheduler.java:468)

I have tried both the settings. Do you just paste them as an extra line under the VM parameters? Like this :

-Xmx3000m
-Xms1000m
-XX:+AggressiveOpts
-Xverify:none
-Dnetbeans.mainclass=org.esa.snap.main.Main
-Dsun.java2d.noddraw=true
-Dsun.awt.nopixfmt=true
-Dsun.java2d.dpiaware=false
snap.gpf.disableTileCache

false
snap.gpf.useFileTileCache

true

1 Like
17

Put the settings on separate lines in $SNAP_HOME/etc/snap.properties or extra lines in the VM parameters setting, but please use the -Dname=value syntax:

...
-Dsun.java2d.dpiaware=false
-Dsnap.gpf.disableTileCache=false
-Dsnap.gpf.useFileTileCache=true
1 Like
18

I experienced the same error “cannot construct data buffer”, but also the “gc overhead limit exceeded” error after I added the band select operator to a previously well running model (using gpt). Actually the band select tool should take the elevation layer from the terrain correction step and export (“write”-tool) the DEM (SRTM 3sec) as geotiff. The changes to snap.properties did not work for me. The more processes I killed to free RAM, the larger the DEM file was created before the crash, but still the reserved 6 GB (through “C:\Program Files\snap\bin\gpt.vmoptions”) were not enough.

I recommend checking the band select tool for optimisation.

19

I was with the same problem when I want to run “S-1 SLC to GRD”. After several hours to several processes and attempts I got these two procedures:
-Update Java to the latest version from the official website.
My notebook:
i72620m 2.28ghz
8gb ram
Windows 10

Image Sentinel 1:
S1A_IW_GRDH_1SDV_20160114T092037_20160114T092112_009488_00DC51_839E (9.23gb)

SNAP/options/

Pasted image384×989 48.4 KB

***these were the three things I did time; but perhaps only one of them the problem is solved.

I hope this is helpful…

1 Like
20

Updating java is probably not one of the helpful things. SNAP come with its own java jre and this is not updated by the normal update process, or have you changed the jre folder in the installation directory?

Most helpful is probably to use the file cache option.