SNAP using S2Agri atmospheric corrections

Hello everyone,

We are using SNAP to compute FAPAR indicator on Sentinel-2 images in parallel with some other S2Agri computations (such as establishing a crop mask).
Since S2Agri already computes the atmospheric correction we’d like to reuse it within SNAP to avoid computing it twice.
This does not seem to be an easy thing, can someone guide us/share some documentation on how to do this?

Here is what we tried so far:

  1. Extract the 8 relevant bands (*) from S2Agri L2A product, stack them with the requested angles (**) coming from L1C product and send the stack to the BioPhysicalOp.
    Snap returns an error: “Bands at 560nm not found”. We believe this error is due to the fact that we directly read the .tif images in L2A product, not a .xml file describing the bands.
    But there is no such .xml file in S2Agri L2A product. So even though we have the good stack, we cannot pass it to the BioPhysicalOp to compute the FAPAR.

  2. We create the exact same folder structure as the one generated by Sen2Cor, and instead of filling them with the Sen2Cor output, we modify the L2A product from Sen2Agri to match Sen2Cor format.
    This should work at the end but it’s very painful!

If you have any suggestion, it will be more than welcome.

Thanks a lot,
Jonathan

(*) B3, B4, B5, B6, B7, B8a, B11, B12
(**) cos(viewing_zenith), cos(sun_zenith), cos(relative_azimuth_angle)

Hello Jonathan.

Unfortunately, at this moment, this is not possible. The Muscate (MACCS) product organisation is different of that of Sen2Cor.
If you’d like to use the Sen2Agri L2A products in SNAP, there should be a specific reader for this purpose. I know that there is some work in progress for this, but the reader is not yet available.

Cheers,
Cosmin.

Hi Jonathan and Cosmin,
we have started developing a converter to use the outputs of MACCS/MAJA in Snap. I’ll try to find some time to put it on a Github tomorrow. But the filling of the xml file is quite some work and is still missing…
Olivier

Hello Olivier.

I think having a converter from MACCS/MAJA to Sen2Cor would be a bit tedious. Instead, a specific SNAP reader would be better. Otherwise, I think you’d have to extract the bands from MAJA rasters and convert them to jp2, and then to write that “nice” SAFE xml format, which is not so nice :slight_smile: . But, if you already started this, maybe you can include the band wavelength information in the metadata. In the current MACCS products (I don’t know if it’s the same for MAJA), this information is not present.

Cosmin.

Hello Olivier and Cosmin,
Thanks a lot for your help.

Olivier, it seems you are doing what we started, I’ll be looking at your Github then. :slight_smile: In the meantime, I am trying to produce the standard .xml file for SNAP, as you said, it’s quite some work.
Please keep us posted on the progress.

Best,
Jonathan

Done, with a very basic description. First users will be beta users…
https://github.com/olivierhagolle/MAJA2SEN2COR

1 Like

That’s great, thanks a lot for sharing. :slight_smile:
So I am probably the beta user. As so, I am sharing some feedback:
It went well until “Process of dimap”, this is the error I get:

################
Process of dimap
################
Read metadata from MACCS product : /home/dial/nico/copiedProducts/S2A_MSIL2A_20170214T080021_N0204_R035_T36NXK_20170214T081513.SAFE
Exception : 'band_id'
Traceback (most recent call last):
  File "MaccsMuscate2Sen2cor.py", line 160, in <module>
    o_product.ConvertDimap()
  File "/home/dial/Documents/algorithms/tests/test_maccs2SC/MAJA2SEN2COR/product/MACCSS2Product.py", line 138, in ConvertDimap
    MACCSS2Dimap.Convert(self.s_productPath, self.s_workingDir)
  File "/home/dial/Documents/algorithms/tests/test_maccs2SC/MAJA2SEN2COR/dimap/MACCSS2Dimap.py", line 272, in Convert
    d_dimapValue, o_Mean_Value_List, o_Sun_Angles_Grids, o_Mean_Sun_Angle, o_Incidence_Angles_Grids_List = cls.ReadMACCSMetadata(_s_productPath)
  File "/home/dial/Documents/algorithms/tests/test_maccs2SC/MAJA2SEN2COR/dimap/MACCSS2Dimap.py", line 103, in ReadMACCSMetadata
    attr_MeanValue = o_Mean_Value.attributes['band_id']
  File "/home/dial/anaconda2/envs/maccs2S2C/lib/python2.7/xml/dom/minidom.py", line 522, in __getitem__
    return self._attrs[attname_or_tuple]
KeyError: 'band_id'
Programme stop

Note that I had to install gdal 2.1.3 to make the imports work.

Thanks for feed back, and transmitted to developers. Please use github “issues” for future remarks
Olivier

Hello jofrisch,

I’m responsable to the developpement of this converter.
This converter permits to:

  • convert L2A products from THEIA format to ESA format, and
  • convert L2A products from MAJA/MACCS/S2AGRI format to ESA format
    The inputs of this converter must be L2A products at MAJA/MACCS/S2AGRI/THEIA format and the output will be L2A products at ESA format.
    Your log seems to indicate that the input is at ESA format (S2A_MSIL2A_20170214T080021* nomenclature). Is it the case?

However, is it possible to continue this discussion by mail?
If yes, please contact us at the mail address indicated at the webpage https://theia.cnes.fr/atdistrib/rocket/#/contact

Hello CDes,
Just sent you an email.
To answer your question my input is indeed a S2A_MSIL2A_20170214T080021* product.
Some context: it’s the result of Sen2Agri’s L2A algo: MACCS. So from what I understand it’s supposed to be a MACCS format.
We can continue this discussion via emails.

Hello jofrisch and CDes,

please don’t hesitate to report your discussion on this forum. I can be interesting for the community.
I will also test the converter soon and will report my results.

Thanks

Hi,

just tested with a MUSCATE product as input.
It ran well and generated jp2 files and a XML file (attached).
However, SNAP cannot open it.
MTD_TL.xml (610.4 KB)

Hi @olivegak971,
I am having the same problem as you, I 've tried to create the .xml to make snap read the file, but without success so far.

On your previous question: we discussed the bug reported above, which is now fixed, and some naming in the tile that should be corrected in a coming update of the github repo. :slight_smile:

Hi all, and Snap developers.
There are a lot of fields in the xml header file of a Sen2cor product. Is there a list of those which are used and read by Snap ? We could then only take care of those ones… We should be able to manage with the reading piece of source code within Snap, but could you find it for us ?

Best regards,
Olivier

Hi @OHagolle,

I could take a look and see what is missing, but it would be easier for me if you can share one example of a converted product.

Regards

Hi Omar, thanks for answering !
@olivegak971 provided one, just above. You can see that it lacks about everything…
That’s why it seemed easier to start from the reading interface of SNAP
Olivier

Actually, there should be a format specification. Which the reader and the writer consider. Think about a third party application (GDAL, Envi) which wants to read the data. If you don’t follow the specs or only satisfy SNAP they are lost.

In the S2 reader of SNAP, we also check the folder structure of the product, it is not enough the metadata file.

Hi Marco,

Of course we have a format specification. Envi, Snap and GDAL are welcome to use it:

Here we are just trying to help some Snap users who would like to use our products providing a converter, with money and time constraints …
Olivier

2 Likes

Hi,

I have made some tests and it seems that there are some things to modify in the converted product to be compatible with the Sentinel-2 reader in SNAP:

1 - Create an empty folder “QI_DATA” at the same level than “IMG_DATA” folder
2 - Modify the metadata xml file:
2.a) The item HORIZONTAL_CS_CODE should contain, before the code, “EPSG:”

2.b) The tag “detector_id” should be replaced by “detectorID”

2.c) The bandID should be an integer from 0 to 12 (in the example, it is used 8A, which is not valid)

2.d) The element TILE_ID_2A should follow the pattern:

(see https://earth.esa.int/web/sentinel/user-guides/sentinel-2-msi/naming-convention)

for example:

I think that is all, if you have still any problem, please tell me.
Regards

1 Like