I try to process S3 data with gpt, mainly for mosaic splicing and output data. I tried to use python to enter the gpt command line, but the result was not good.
Below is my xml file and python file
Thank you！
myGraph.xml (8.1 KB) S3.py (1.1 KB)
What was not good with the result?
By having a quick look at your files, I saw that you specify input1 and input2 in your python code, but you use only the variable input in the xml file. Maybe this is the issue?
Thank you for your reply！
Sorry I didn’t express it clearly，it means my code cannot work
I chang the variable into input1 and input2 in the xml file,but it still does’t work.
Maybe the problem lies in my xml file, but I can’t see what’s wrong.
here is the result
Have you tried the generated command manually on the command line?
This will give you more information about the problem.
And add the -e option. This will generate detailed error messages.
Thank you !
Follow your advice,I fix the xml file ,now it work
sorry,An error occurred in the code during runtime,here is the running log.
I can’t find out what caused this error, can I trouble you to answer it? Thank you！
PS C:\Program Files\snap\bin> gpt D:\s_data\code\myGraph.xml -Pinput1=D:\s_data\sentinel\ex\S3A_OL_2_WFR____20180412T015441_20180412T015741_20180412T035135_0179_030_060_2340_MAR_O_NR_002.SEN3\xfdumanifest.xml -Pinput2=D:\s_data\sentinel\ex\S3A_OL_2_WFR____20180412T015741_20180412T020041_20180412T035521_0179_030_060_2520_MAR_O_NR_002.SEN3\xfdumanifest.xml -Poutput=D:\s_data\sentinel\output\20180412T015441_Sentinel_3_ECS.h5
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.esa.snap.core.util.EngineVersionCheckActivator: Please check regularly for new updates for the best SNAP experience.
Executing processing graph
INFO: org.hsqldb.persist.Logger: dataFileCache open start
90% done.
org.esa.snap.core.gpf.OperatorException: java.lang.IllegalStateException: java.util.zip.ZipException: incorrect data check
at org.esa.snap.core.gpf.graph.GraphProcessor$GPFImagingListener.errorOccurred(GraphProcessor.java:363)
at com.sun.media.jai.util.SunTileScheduler.sendExceptionToListener(SunTileScheduler.java:1646)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:921)
at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at javax.media.jai.PointOpImage.computeTile(PointOpImage.java:911)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at javax.media.jai.RenderedOp.getTile(RenderedOp.java:2257)
at javax.media.jai.PointOpImage.computeTile(PointOpImage.java:911)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at com.sun.media.jai.iterator.RandomIterFallback.makeCurrent(RandomIterFallback.java:108)
at com.sun.media.jai.iterator.RandomIterFallback.getSampleFloat(RandomIterFallback.java:125)
at com.sun.media.jai.opimage.WarpNearestOpImage.computeRectFloat(WarpNearestOpImage.java:424)
at com.sun.media.jai.opimage.WarpNearestOpImage.computeRect(WarpNearestOpImage.java:111)
at javax.media.jai.WarpOpImage.computeTile(WarpOpImage.java:577)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at javax.media.jai.RenderedOp.getTile(RenderedOp.java:2257)
at javax.media.jai.PointOpImage.computeTile(PointOpImage.java:911)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
at org.esa.snap.core.image.VirtualBandOpImage.addDataToReferredRasterDataSymbols(VirtualBandOpImage.java:309)
at org.esa.snap.core.image.VirtualBandOpImage.computeTile(VirtualBandOpImage.java:235)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
at javax.media.jai.PlanarImage.getExtendedData(PlanarImage.java:2440)
at com.sun.media.jai.opimage.MosaicOpImage.computeTile(MosaicOpImage.java:432)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
at javax.media.jai.RenderedOp.getData(RenderedOp.java:2276)
at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:73)
at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at com.sun.media.jai.util.RequestJob.compute(SunTileScheduler.java:247)
at com.sun.media.jai.util.WorkerThread.run(SunTileScheduler.java:468)
Caused by: java.lang.RuntimeException: java.lang.IllegalStateException: java.util.zip.ZipException: incorrect data check
at org.esa.s3tbx.dataio.s3.util.S3VariableOpImage.computeRect(S3VariableOpImage.java:105)
at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
… 40 more
Caused by: java.lang.IllegalStateException: java.util.zip.ZipException: incorrect data check
at ucar.nc2.iosp.LayoutBBTiled.hasNext(LayoutBBTiled.java:134)
at ucar.nc2.iosp.hdf5.H5tiledLayoutBB.hasNext(H5tiledLayoutBB.java:126)
at ucar.nc2.iosp.IospHelper.readData(IospHelper.java:324)
at ucar.nc2.iosp.IospHelper.readDataFill(IospHelper.java:292)
at ucar.nc2.iosp.hdf5.H5iosp.readData(H5iosp.java:163)
at ucar.nc2.iosp.hdf5.H5iosp.readData(H5iosp.java:141)
at ucar.nc2.NetcdfFile.readData(NetcdfFile.java:1957)
at ucar.nc2.Variable.reallyRead(Variable.java:879)
at ucar.nc2.Variable._read(Variable.java:865)
at ucar.nc2.Variable.read(Variable.java:677)
at org.esa.s3tbx.dataio.s3.util.S3VariableOpImage.computeRect(S3VariableOpImage.java:102)
… 42 more
Caused by: java.util.zip.ZipException: incorrect data check
at java.util.zip.InflaterInputStream.read(Unknown Source)
at java.io.FilterInputStream.read(Unknown Source)
at ucar.nc2.util.IO.copy(IO.java:104)
at ucar.nc2.iosp.hdf5.H5tiledLayoutBB$DataChunk.inflate(H5tiledLayoutBB.java:218)
at ucar.nc2.iosp.hdf5.H5tiledLayoutBB$DataChunk.getByteBuffer(H5tiledLayoutBB.java:192)
at ucar.nc2.iosp.LayoutBBTiled.hasNext(LayoutBBTiled.java:128)
… 52 more
Error: java.lang.IllegalStateException: java.util.zip.ZipException: incorrect data check
Try like this (you will have to adapt it to Windows, I think you need to use quotes " " in your directories):
gpt /Users/Anita/Desktop/SNAP_Forum/gpt_manacapuru.xml -t /Users/Anita/Desktop/SNAP_Forum/Downloads/S3A_OL_2_WFR____20191104T135002_20191104T135302_20191124T134508_0179_051_124_3060_MAR_O_NT_002_subset.dim -Ssource=/Users/Anita/Desktop/SNAP_Forum/Downloads/S3A_OL_2_WFR____20191104T135002_20191104T135302_20191124T134508_0179_051_124_3060_MAR_O_NT_002.SEN3
Sorry i am a beginner，I tried putting quotes on my path And add Ssource after each input，but the error still exists. The command like that
gpt D:\s_data\code\myGraph.xml -Pinput1="D:\s_data\sentinel\ex\S3A_OL_2_WFR____20180412T015441_20180412T015741_20180412T035135_0179_030_060_2340_MAR_O_NR_002.SEN3\xfdumanifest.xml " -Pinput2=“D:\s_data\sentinel\ex\S3A_OL_2_WFR____20180412T015741_20180412T020041_20180412T035521_0179_030_060_2520_MAR_O_NR_002.SEN3\xfdumanifest.xml” -Poutput=“D:\s_data\sentinel\output\20180412T015441_Sentinel_3_ECS.h5”
gpt D:\s_data\code\myGraph.xml -Pinput1="D:\s_data\sentinel\ex\S3A_OL_2_WFR____20180412T015441_20180412T015741_20180412T035135_0179_030_060_2340_MAR_O_NR_002.SEN3\xfdumanifest.xml " -Ssource=D:\s_data\sentinel\ex\S3A_OL_2_WFR____20180412T015441_20180412T015741_20180412T035135_0179_030_060_2340_MAR_O_NR_002.SEN3 -Pinput2=“D:\s_data\sentinel\ex\S3A_OL_2_WFR____20180412T015741_20180412T020041_20180412T035521_0179_030_060_2520_MAR_O_NR_002.SEN3\xfdumanifest.xml” -Ssource=D:\s_data\sentinel\ex\S3A_OL_2_WFR____20180412T015441_20180412T015741_20180412T035135_0179_030_060_2340_MAR_O_NR_002.SEN3 -Pinput2="D:\s_data\sentinel\ex\S3A_OL_2_WFR____20180412T015741_20180412T020041_20180412T035521_0179_030_060_2520_MAR_O_NR_002.SEN3 -Poutput=“D:\s_data\sentinel\output\20180412T015441_Sentinel_3_ECS.h5”
The error seems to indicate a problem with the data. Can you open the product in SNAP Desktop?
In addition, the stack trace seems not to match the current source code.
Are you still using SNAP 7?
I update to SNAP 8.0 and I can open those product in SNAP Desktop, the data looks good, but I can’t use GUI to mosaic either,Could there be a problem with the data? Is there any way to solve it? .
Thank you for your reply！
What is not working when you try to mosaic in the GUI. Do you get the same error?
Have you update to SNAP 8 just now or already before? If you did it before I can’t explain the stack trace.
Can you provide the 2 data products?
I will let you know a location where you can upload them in a separate message.
I already update to SNAP 8 and get same error in GUI: incorrect data check
I can provide the 2 data products, I down load them from https://archive.eumetsat.int/
They are S3A_OL_2_WFR____20180412T015741_20180412T020041_20180412T035521_0179_030_060_2520_MAR_O_NR_002.SEN3
and
S3A_OL_2_WFR____20180412T015441_20180412T015741_20180412T035135_0179_030_060_2340_MAR_O_NR_002.SEN3
Your graph.xml file you have attached in you first post had some more issues.
I’ve updated it and attached it. myGraph.xml (10.0 KB)
With this I was able to create the mosaic.
The command line call is added in the header of the file as comment.
As a note, you should only enable the orthorectification when you use the tie-point geo-coding for the OLCI products. If you use the per-pixel geo-coding you do the orthorectification twice which will lead to wrong results.
See also: S3TBX FAQs - SNAP - SNAP Wiki (atlassian.net)
I’ve created also a simpler version. Reader and writer nodes are not necessary.
myGraph2.xml (9.4 KB).
With this you can easily extend the number of products to be mosaicked.
How the command line call looks like is also added as comment in the header.
I would also suggest not to use hdf5, better NetCDF4-CF. Writing the data is faster and it takes less disk space (7.3GB vs. 470MB).
Thank for your reply! That’s help a lot!