Abnomalies in sen2cor execution and L2A_Tables.py file

Optical Toolbox sen2cor
#1

Hello,

All users of Sentinel2 imagery need to use sen2cor to process it from level L1C to L2A. Not only there are several problems with sen2cor installation but apparently there are also several bugs.

@ChristianSeverin has brilliantly shown there’s a bug in the L2A_Tables.py and he offered a solution. Since I’m not familiar with python, I copied the file version as kindly uploaded in this forum by @riyaadh91. After that I did the following tests:

  1. Run sen2cor on a whole Sentinel-2 image

  2. Run sen2cor on selected granules/tiles of a Sentinel-2 image

As explained by ChristianSeverin I am using the command line to execute sen2cor. This is faster, easy and allows to follow all the process.

I have found the following:

  1. You can process only selected granules if you keep them in the full image directory (you eliminate the granules you don’t need) probably because the image xml file contains information not available inside the xml files of the granules.

  2. Whatever the method (full image or selected tiles), the same validity issues occurr with the xml files generated by sen2cor itself:

  • Files S2A_USER_MTD_SAFL2A_PDMC_…xml are systematically evaluated as invalid

  • Files S2A_USER_MTD_L2A_TL_SGS_…xml are systematically evaluated as invalid

  • Files S2A_USER_MTD_L2A_DS_SGS_…xml are systematically evaluated as valid

  • Also there are some strange messages but target already exists while copying different files that sen2cor itself has created!

The following image shows a sample of this phenomenon that happens with each granule.

sen2cor errors.jpg1160×145 100 KB

However I could get processed images of all selected granules and also of full image. However, when I produced RGBs in SNAP, some of the processed tiles have different colours with seams between them whereas the L1C does not have any seams.

The following images show the situation before and after processing:

Shashamene_L1C.jpg642×781 317 KB

Shashamene_L2A.jpg637×785 361 KB

I would like to ask if:

  • @ChristianSeverin could kindly make available (upload) his corrected L2A_Tables.py as many of us have little command of python and also because he perhaps has made new corrections that we can test.

  • Someone has encountered similar problems, has any explanations and/or solutions?

Kind regards to all.

#2

Sorry, @Ghanisen, all I did with L2A_Tables.py was adding those few lines to fix the copy operations …

#3

Dear @ChristianSeverin,

Could you kindly upload it in the forum because that would avoid dangerous code manipulations to people like me who are unfamiliar with python?

Kind regards.

#4

@Ghanisen, it seems that the broken copy_file() method call was removed from the L2A_Tables.py file in the GitHub repository as part of a major rewrite somewhen in March.

However, while today’s (2016-04-14) Sen2Cor version on GitHub is 2.2.0, the download page at telespazio still has version 2.0.6 only …

Correction: an (experimental?) download for version 2.2.0 can be found in folder test_new_setup_win! Yay!
So you could either fix the old L2A_Tables.py, or download the 2.2.0 beta.

1 Like
#5

Thanks @ChristianSeverin! very good news indeed.

Best regards.

#6

Dear @ChristianSeverin

Could you install the new sen2cor? I have syntax errors while executing ‘set_env’ as specified in their readme file:

new sencor installation errors.jpg1287×467 185 KB

Do you have an idea about this?

Kind regards.

#7

Concerning these off-color tile artefacts: according to Bringfried Pflug (bdpg), they are due to the fact that “the processing baseline of Sen2Cor is procesing L1C products tile by tile, with each tile processed independent from the others. Each tile is different and gives different aerosol optical thickness. This results in differences in the final BOA-product”.

1 Like
#8

Thanks to you @ChristianSeverin and to @bdpg for this information.

Did you manage to install sen2cor 2.2?

Kind regards.

#9

Haven’t tried yet, sorry.

#10

Hi @ChristianSeverin, Hi @bdpg

I have first downloaded and unzipped the file sen2cor-2.2.0.zip using your link. Then I followed their instructions to set the environment variables; it didn’t work as some variables (e.g. for GDAL) were not set.

I then removed manually the varibales that were added, copied the unzipped file in the root directory of my C: drive and repeated the set_env execution. This time around it worked perfectly and all variables were set properly.

I then processed 4 contiguous L1C granules pertaining to the same image. There were absolutely no errors mentioned during execution of the L2A_Process and the resulting L2A products were OK but presented the same color differences between some tiles due the same reasons @bdpg has mentioned.

I am now trying to see if I can do the color balancing in OTB-Monteverdi 3.0.1 and, if it doesn’t work, I’ll try with IDRISI TerrSet. I’ll let you know the results.