S1A_IW_SLC__1SDV_ thermal noise Error

saiplanner · September 30, 2018, 2:17pm

I am trying to run the xml file from gpt in HPC. The SNAP software is updated. For the products acquired before March 13, 2018 are working fine.

The following error occurs.
[pratiman@master1 Sigma0]$ gpt Top_S1_20180318_Orb_TNoise_Cal_Deb_ML_TC.xml
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
SEVERE: org.esa.s2tbx.dataio.gdal.activator.GDALDistributionInstaller: The environment variable LD_LIBRARY_PATH does not contain the current folder ‘.’. Its value is ‘/home/pratiman/java_8_131/jre1.8.0_131/lib:/home/pratiman/lis/lib/lib:/home/pratiman/intel/libs/lib:/opt/hdf5/lib/:/opt/gcc-4.8.2/lib/:/opt/gcc-4.8.2/lib64/:/lscratch/intel/impi/5.0.1.035/intel64/lib:/lscratch/intel/composer_xe_2015.0.090/compiler/lib/intel64:/lscratch/intel/composer_xe_2015.0.090/mpirt/lib/intel64:/lscratch/intel/composer_xe_2015.0.090/ipp/…/compiler/lib/intel64:/lscratch/intel/composer_xe_2015.0.090/ipp/lib/intel64:/lscratch/intel/composer_xe_2015.0.090/compiler/lib/intel64:/lscratch/intel/composer_xe_2015.0.090/mkl/lib/intel64:/lscratch/intel/composer_xe_2015.0.090/tbb/lib/intel64/gcc4.4:/lscratch/intel/composer_xe_2015.0.090/compiler/lib/intel64:/lscratch/intel/composer_xe_2015.0.090/mkl/lib/intel64:/lscratch/intel/composer_xe_2015.0.090/compiler/lib/intel64:/lscratch/intel/composer_xe_2015.0.090/mpirt/lib/intel64:/lscratch/intel/composer_xe_2015.0.090/ipp/…/compiler/lib/intel64:/lscratch/intel/composer_xe_2015.0.090/ipp/lib/intel64:/lscratch/intel/composer_xe_2015.0.090/compiler/lib/intel64:/lscratch/intel/composer_xe_2015.0.090/mkl/lib/intel64:/lscratch/intel/composer_xe_2015.0.090/tbb/lib/intel64/gcc4.4:/usr/local/munge/lib:/lscratch/PGI/linux86-64/17.1/lib:/lscratch/PGI/linux86-64/2017/mpi/openmpi-1.10.2/lib::/home/pratiman/local/lib’.
Executing processing graph
INFO: org.hsqldb.persist.Logger: dataFileCache open start
java.lang.NullPointerException
done.

Error: [NodeId: ThermalNoiseRemoval] java.lang.NullPointerException

saiplanner · September 30, 2018, 2:45pm

I have run the same file in Windows PC where the SNAP is up to date. The following error occurs

Picked up _JAVA_OPTIONS: -Xmx1536M
INFO: org.esa.snap.python.gpf.PyOperatorSpi: Python operator ‘py_sambuca_snap_op’ registered (Python module: ‘sambuca_snap_op’, class: ‘sambuca_snap_op’, root: ‘C:\Users\VVSK\AppData\Roaming\SNAP\modules\org-esa-sen2coral-sen2coral-inversion.jar’)
INFO: org.esa.snap.python.gpf.PyOperatorSpi: Python operator ‘S2RutOp’ registered (Python module: ‘s2_rut’, class: ‘S2RutOp’, root: ‘C:\Users\VVSK\AppData\Roaming\SNAP\modules\org-esa-snap-snap-rut.jar’)
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
Executing processing graph
INFO: org.hsqldb.persist.Logger: dataFileCache open start
java.lang.NullPointerException
java.lang.NullPointerException
java.lang.NullPointerException
java.lang.NullPointerException
90% done.
org.esa.snap.core.gpf.OperatorException: Cannot construct DataBuffer.
at org.esa.snap.core.gpf.graph.GraphProcessor$GPFImagingListener.errorOccurred(GraphProcessor.java:363)
at com.sun.media.jai.util.SunTileScheduler.sendExceptionToListener(SunTileScheduler.java:1646)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:921)
at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
at org.esa.s1tbx.sentinel1.gpf.TOPSARDeburstOp.computeTileInOneSwathFloat(TOPSARDeburstOp.java:895)
at org.esa.s1tbx.sentinel1.gpf.TOPSARDeburstOp.computeTileStack(TOPSARDeburstOp.java:802)
at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeRect(OperatorImageTileStack.java:116)
at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeTile(OperatorImageTileStack.java:85)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
at org.esa.s1tbx.sar.gpf.MultilookOp.computeTile(MultilookOp.java:176)
at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.getPixelValue(RangeDopplerGeocodingOp.java:1242)
at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.computeTileStack(RangeDopplerGeocodingOp.java:1052)
at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeRect(OperatorImageTileStack.java:116)
at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeTile(OperatorImageTileStack.java:85)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:73)
at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at com.sun.media.jai.util.RequestJob.compute(SunTileScheduler.java:247)
at com.sun.media.jai.util.WorkerThread.run(SunTileScheduler.java:468)
Caused by: java.lang.RuntimeException: Cannot construct DataBuffer.
at com.sun.media.jai.util.DataBufferUtils.constructDataBuffer(DataBufferUtils.java:132)
at com.sun.media.jai.util.DataBufferUtils.createDataBufferFloat(DataBufferUtils.java:214)
at javax.media.jai.ComponentSampleModelJAI.createDataBuffer(ComponentSampleModelJAI.java:271)
at javax.media.jai.RasterFactory.createWritableRaster(RasterFactory.java:691)
at javax.media.jai.PlanarImage.createWritableRaster(PlanarImage.java:1982)
at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:130)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
… 46 more

Error: Cannot construct DataBuffer.

mengdahl · October 1, 2018, 6:52am

If you have updated SNAP to the latest version the error could be related to your settings - what do you have in the JVM options, tile cache setting etc.?

saiplanner · October 2, 2018, 5:08am

From the previous post, I understood that the modules are updated only for GRDH images not for SLC.
In my windows pc (16 gb ram, i7 processor) I configured the snap settings as follows:
Cache size: 8448 mb, JVM options: -Xmx13G

marpet · October 2, 2018, 6:03am

Where is this line coming from?
1.5 GB is not enough.

saiplanner · October 2, 2018, 7:02pm

I run my xml file from SNAP Command Line as usual. I never noticed this. It automatically came I didn’t do anything.

mfitrzyk · October 8, 2018, 1:45pm

So I understand that your workflow for product before March 13th works fine and it does not for products after March?
Do you use thermal noise removal ?

saiplanner · October 8, 2018, 1:48pm

Yes. Without Thermal Noise Removal it is working fine.

mfitrzyk · October 8, 2018, 2:03pm

Go in SNAP to help\About SNAP and check the version. Than go to check for updates and update the software.
It might be that you don’t have all the updates because in the current version you can apply thermal noise removal to all datasets - before and after March 2018.

saiplanner · October 9, 2018, 11:02am

Hi,
I tried in my windows PC it is working. But the problem is it not working in HPC. It might be because the HPC has no internet connection and so the updates might have not installed. The following is the configuration in HPC.

Any clue about how to install the updates offline?

Thanking you,

mfitrzyk · October 11, 2018, 1:35pm

yes, indeed you have the latest version but without current updates.
The TNR operator for the data acquired after March 2018 was released in one of the updates.
Downloading the latest updates shall solve your problem.

mengdahl · October 11, 2018, 4:39pm

@TonioF @obarrilero any ideas on how to distribute updated SNAP to a machine with no internet connection?

obarrilero · October 12, 2018, 7:54am

Hi,

I think that there are two options:

Download the nbm’s from the update centers (you can see the URL in SNAP->Tools->Plugin->Settings) and install them via SNAP->Tools->Plugin->Downloaded-> Add plugins.
Download everything in the update center (including the xml files), copy it to a local folder and then create a local update center indicating as URL your local folder (file:///[yourPath]). You can do it in SNAP->Tools->Plugin->Settings->Add