Sentinel 3 Forel-ULe GPT errors

Hello,
I run FU-Classification from SNAP using Sentinel 3 Water Full Product downloaded from EUMETSAT in an automated process:

  1. gpt_processing cmd.pygpt_processing_cmd.py (1.8 KB)
  • which loops through a list of data and calls the cmd and the GPT graph
    2.fuGraph.xml
    fuGraph.xml (2.4 KB)

I ran the process for 2017,2018, 2019 and 2020 and I sometimes encounter the following errors:

“Error: [NodeId: fuNode] The instrument can not be automatically detected, please select the instrument in the processing parameter.” This is the error I get from running the script. When I run the same process in SNAP I get:
image

I also get a lot of “Cannot construct DataBuffer.”, and reading the other posts in the forum, it seems like it relates to the memory. I have 32GB RAM and 8 cores so this should not be the issue. The Picked up _JAVA_OPTIONS are -Xms256m -Xmx1024m Is it possible if this error occurs if there is no valid water data after subsetting S3 in the node id= “Subset”?

Processing…
Picked up _JAVA_OPTIONS: -Xms256m -Xmx1024m
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Incompatible GDAL 3.3.1 found on system. Internal GDAL 3.0.0 from distribution will be used.
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Internal GDAL 3.0.0 set to be used by SNAP.
INFO: org.esa.snap.core.util.EngineVersionCheckActivator: Please check regularly for new updates for the best SNAP experience.
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Internal GDAL 3.0.0 set to be used by SNAP.
INFO: org.hsqldb.persist.Logger: dataFileCache open start
org.esa.snap.core.gpf.OperatorException: Cannot construct DataBuffer.
at org.esa.snap.core.gpf.graph.GraphProcessor$GPFImagingListener.errorOccurred(GraphProcessor.java:379)
at com.sun.media.jai.util.SunTileScheduler.sendExceptionToListener(Unknown Source)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(Unknown Source)
at javax.media.jai.OpImage.getTile(Unknown Source)
at com.sun.media.jai.iterator.RandomIterFallback.makeCurrent(Unknown Source)
at com.sun.media.jai.iterator.RandomIterFallback.getSampleFloat(Unknown Source)
at com.sun.media.jai.opimage.WarpNearestOpImage.computeRectFloat(Unknown Source)
at com.sun.media.jai.opimage.WarpNearestOpImage.computeRect(Unknown Source)
at javax.media.jai.WarpOpImage.computeTile(Unknown Source)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(Unknown Source)
at javax.media.jai.OpImage.getTile(Unknown Source)
at javax.media.jai.PlanarImage.getData(Unknown Source)
at javax.media.jai.RenderedOp.getData(Unknown Source)
at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:449)
at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:435)
at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeRect(OperatorImageTileStack.java:116)
at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeTile(OperatorImageTileStack.java:86)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(Unknown Source)
at javax.media.jai.OpImage.getTile(Unknown Source)
at com.sun.media.jai.util.RequestJob.compute(Unknown Source)
at com.sun.media.jai.util.WorkerThread.run(Unknown Source)
Caused by: java.lang.RuntimeException: Cannot construct DataBuffer.
at com.sun.media.jai.util.DataBufferUtils.constructDataBuffer(Unknown Source)
at com.sun.media.jai.util.DataBufferUtils.createDataBufferFloat(Unknown Source)
at javax.media.jai.ComponentSampleModelJAI.createDataBuffer(Unknown Source)
at javax.media.jai.RasterFactory.createWritableRaster(Unknown Source)
at javax.media.jai.PlanarImage.createWritableRaster(Unknown Source)
at javax.media.jai.SourcelessOpImage.computeTile(Unknown Source)
… 20 more

Error: Cannot construct DataBuffer.

Any suggestions, please?

SNAP FAQs - SNAP - Confluence (atlassian.net)

You say that you have 32GB of RAM, but you limit it to 1GB by setting -Xmx1024m
But I don’T know whre you set this.
By default this value is set to ~3/4 of the available RAM.

The autodetection chechs for at least one existing band following this pattern: “Oa\d+_reflectance”
and the product type needs to contain OL_2 (this type is shown in the information dialog).

Maybe the data wasn’t properly downloaded?
Instead of using the autodetection you could set the instrument explicitly to OLCI.

Many thanks to your replies! @marpet I believe the java options are set up in my environment variables:


What would be your recommendation to change these values to, please?
My system (you might notice it is a virtual machine this is run on):

Also, following from the post here https://forum.step.esa.int/t/snappy-ini-not-found-make-new-one/7422: I can try to create the snappy.ini but would it be better to change the java specs in the environment variables instead? It seems like the file is missing from my snappy folder.

I changed the “AUTO_DETECT” in the graph to “OLCI”, but it still throws and error for some of the images. The product (OL2 product with “Oa\d+_reflectance”)


And the error message:

Apologies if this is obvious but all advice is much appreciated!

I would suggest to delete the _JAVA_OPTIONS variable or your rename it, then you can easily reactivate it if necessary. It could be that some other application which has set this variable might have issues.
But it is not a good habit to set this variable. Because it has an impact on all Java application.
For some settings this might be useful because they are dependent on the system and all Java applications need to be configured this way. But to limit the maximum memory is a no go.

Here you can find a discussions about it: