Hi S2tbx developers,
I have seen in the release notes of SNAP 6.0.1 updates that a MUSCATE reader for Theia L2A products (https://theia.cnes.fr) generated with MAJA has been added. That’s great, thanks for that !
Where could I find some documentation about it ?
Best regards,
Olivier
Just to put my question on top of the list 
@obarrilero, is there any documentation ?
Thanks !
Olivier
Salut Olivier,
sorry for the late reply. The documentation is not yet available, but we have used the product description available in the link below (I am sure that you know it very well). If you are interested in some particular point, please, do not hesitate to ask me and I will try to clarify you how it has been implemented.
https://theia.cnes.fr/atdistrib/rocket/#/help
Some details of the current implementation:
Of course, any comment or suggestion is welcome. For example, do you think that it would be better to use a property for opening only the reflectances with or without slope correction instead of opening both? Is the reader missing any important information?
Regards,
Omar.
Hi Omar,
I am not much a SNAP user. My intention was to make an announcement of the availability of that plugin on CESBIO’s blog, so that users know this solution is available. I also wanted to include a link to some documentation, but if does not exist, I’ll just make the announcement.
Thanks, anyway !
I think it is OK to open both types of products.
Best regards,
Olivier
Dear developers,
I am trying to use the S2Resampling Processor on a Sentinel-2 Theia L2A product.
Unfortunately the operator gives the error ‘Invalid S2 source product’.
Are you planning to make the products opened using the Sentinel-2 Toolbox MUSCATE Reader compatible with the S2Resampling operator in the future releases of SNAP?
I think that the proposal by @obarrilero for opening only the reflectances with or without slope correction instead of opening both could help to make the product compatible with the operator.
Regards,
Federico
Hi @FedericoF,
Thank you for your suggestion, I have just added this improvement to our issue tracker. I will try to include it in the next release.
Regards,
Omar.
Hi @obarrilero,
thank you for considering my suggestion.
I have a new issue related to the Sentinel-2 Toolbox MUSCATE Reader.
The different versions of Sentinel-2 Theia L2A products (v1.4 and v1.7) contain different sets of masks.
I have noted that the Sentinel-2 Toolbox MUSCATE Reader is not importing some of the masks for older v1.4 products, even if they exists in the product (e.g. masks ‘MG2’). Can this behaviour be checked?
Regards,
Federico
Hi @FedericoF,
yes, this will be checked. In fact, the problem that we had in the S2Resampling is that we needed some masks that were not present in the old products (the detector footprints).
Regards,
Omar.
In addition to the abovementioned issues, I have noticed that some MUSCATE products can not be opened in SNAP by directly importing the ZIP file. The error ‘No appropriate product reader found’ is reported. When the same product is unzipped it can be opened without any problem.
It does not seem to be related to a specific product version, and occurs for example for product:
SENTINEL2A_20170315-101214-518_L2A_T32TPR_D.zip
Hope this may help to find the cause of the problem.
Federico
Dear SNAP developers, dear @obarrilero
thank you for updating the Sentinel-2 Toolbox MUSCATE Reader.
The ‘S2Resampling’ operator can now be successfully applied to the MUSCATE Sentinel-2 L2A products, and the masks of v1.4 products are correctly imported.
Nevertheless, using the last SNAP updates, I am experiencing errors while opening the MUSCATE products. I have done a statistical analysis over the errors and discovered that they comes from three different clustered temporal periods.
Could the error be related to some modifications in the MUSCATE product format or generation procedure?
I hope this could be solved in the next SNAP release.
Error related to the image cluster belonging to period ‘2016-12 - 2017-07’ and period ‘2019-02 - 2019-03’ is:
Invalid S2 source product
Error related to the image cluster belonging to period ‘2018-05 - 2018-06’ is the following:
SEVERE: org.esa.s2tbx.dataio.gdal.activator.GDALPlugInActivator: Failed to delete the GDAL distribution folder ‘/home/prepro/.snap6/auxdata/gdal’.
java.lang.IllegalArgumentException: Failed to delete the GDAL distribution folder ‘/home/prepro/.snap6/auxdata/gdal’.
at org.esa.s2tbx.dataio.gdal.GDALInstaller.copyDistribution(GDALInstaller.java:92)
at org.esa.s2tbx.dataio.gdal.activator.GDALDistributionInstaller.install(GDALDistributionInstaller.java:56)
at org.esa.s2tbx.dataio.gdal.activator.GDALPlugInActivator.start(GDALPlugInActivator.java:22)
at org.esa.snap.runtime.Engine.informActivators(Engine.java:222)
at org.esa.snap.runtime.Engine.lambda$start$0(Engine.java:121)
at org.esa.snap.runtime.Engine.runClientCode(Engine.java:189)
at org.esa.snap.runtime.Engine.start(Engine.java:121)
at org.esa.snap.runtime.Engine.start(Engine.java:90)
at org.esa.snap.runtime.Launcher.run(Launcher.java:51)
at org.esa.snap.runtime.Launcher.main(Launcher.java:31)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.exe4j.runtime.LauncherEngine.launch(LauncherEngine.java:62)
at com.install4j.runtime.launcher.UnixLauncher.main(UnixLauncher.java:57)INFO: org.hsqldb.persist.Logger: dataFileCache open start
java.lang.NullPointerException
at org.esa.s2tbx.dataio.muscate.MuscateMetadata.getImages(MuscateMetadata.java:167)
at org.esa.s2tbx.dataio.muscate.MuscateProductReader.readProductNodesImpl(MuscateProductReader.java:138)
at org.esa.snap.core.dataio.AbstractProductReader.readProductNodes(AbstractProductReader.java:169)
at org.esa.snap.core.gpf.common.ReadOp.initialize(ReadOp.java:116)
at org.esa.snap.core.gpf.internal.OperatorContext.initializeOperator(OperatorContext.java:486)
at org.esa.snap.core.gpf.internal.OperatorContext.getTargetProduct(OperatorContext.java:273)
at org.esa.snap.core.gpf.Operator.getTargetProduct(Operator.java:387)
at org.esa.snap.core.gpf.graph.NodeContext.initTargetProduct(NodeContext.java:77)
at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:195)
at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:178)
at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:178)
at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:178)
at org.esa.snap.core.gpf.graph.GraphContext.initOutput(GraphContext.java:162)
at org.esa.snap.core.gpf.graph.GraphContext.(GraphContext.java:91)
at org.esa.snap.core.gpf.graph.GraphContext.(GraphContext.java:64)
at org.esa.snap.core.gpf.graph.GraphProcessor.executeGraph(GraphProcessor.java:128)
at org.esa.snap.core.gpf.main.DefaultCommandLineContext.executeGraph(DefaultCommandLineContext.java:86)
at org.esa.snap.core.gpf.main.CommandLineTool.executeGraph(CommandLineTool.java:534)
at org.esa.snap.core.gpf.main.CommandLineTool.runGraph(CommandLineTool.java:388)
at org.esa.snap.core.gpf.main.CommandLineTool.runGraphOrOperator(CommandLineTool.java:287)
at org.esa.snap.core.gpf.main.CommandLineTool.run(CommandLineTool.java:188)
at org.esa.snap.core.gpf.main.CommandLineTool.run(CommandLineTool.java:121)
at org.esa.snap.core.gpf.main.GPT.run(GPT.java:54)
at org.esa.snap.core.gpf.main.GPT.main(GPT.java:34)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.esa.snap.runtime.Launcher.lambda$run$0(Launcher.java:55)
at org.esa.snap.runtime.Engine.runClientCode(Engine.java:189)
at org.esa.snap.runtime.Launcher.run(Launcher.java:51)
at org.esa.snap.runtime.Launcher.main(Launcher.java:31)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.exe4j.runtime.LauncherEngine.launch(LauncherEngine.java:62)
at com.install4j.runtime.launcher.UnixLauncher.main(UnixLauncher.java:57)
Hi @FedericoF,
please, could you tell me the name of some of the problematic products in order to try to reproduce the error?
Dear @obarrilero,
here a list of products for the three periods:
SENTINEL2A_20170606-102456-040_L2A_T32TPP_D
SENTINEL2A_20180529-101826-293_L2A_T32TPP_D
SENTINEL2B_20180613-101424-177_L2A_T32TPP_D
SENTINEL2A_20190223-101751-925_L2A_T32TQS_D
SENTINEL2B_20190310-101753-474_L2A_T32TQS_D
Federico
Hi @FedericoF,
I have downloaded some of the products and I have found different problems:
Some of the zip files cannot be opened directly from SNAP (we have to investigate it, perhaps the parameters of compression have changed), but if you unzip the products, you can open them in SNAP.
In the product from 2017, the detector footprint masks are not included, so it is not possible to apply the S2Resampling operator.
In at least one of the products from 2019, there is an error in the metadata. The paths to the footprint masks are wrong (it seems that it has been changed L2A by L1C), so SNAP cannot find them, but you can edit manually the metadata file.
Regards,
Omar.
Thanks Omar for this report. We have had some changes with time of the product format, but the last point on detector footprints is a known bug which has been corrected with version 3.2.2 (which is not not production yet at Theia, but is available for PEPS on demand processing).
Sorry for the complexity.
Olivier
Thanks @obarrilero and @OHagolle for reporting the problems and highlighting the changes done on MUSCATE products.
I will wait for the new SNAP release in order to read some of the problematic products for 2018.
Thanks Olivier for suggesting the use of PEPS on demand processing to get latest version products and overcome one of the problems.
Regarding the error in the metadata, I have temporarily fixed it by using the command ‘sed’ to edit the wrong lines with the mask file paths. I share my workaround in case any other user may need it:
sed -i -e ‘/^.*MASK_FILE/s/_L1C_/_L2A_/g’ metadata_MTD_ALL.xml
Federico
Hi @obarrilero,
I have just found a new issue on Sentinel-2 data in MUSCATE format downloaded from Theia.
Data acquired after the end of February 2019 are imported in SNAP, after the ‘L2A’ fix in the ‘MTD_ALL.xml’ file. The masks belonging to group ‘MG2’ are not imported in SNAP, and this seems related to a change in the xml file.
Hope this can be fixed in the next release.
Federico
Hi @obarrilero,
I am facing problems importing the new Sentinel-2 L2A in MUSCATE format generated from PEPS on demand processing. The problem is related to the masks belonging to group ‘MG2’, that are not imported in SNAP.
Could you also have a look at this when fixing the problem for the Sentinel-2 MUSCATE reader (Theia catalogue) for images acquired after the end of February 2019?
Thank you for the support,
Federico
