Sen2cor 2.4.0. parsing errors

Optical Toolbox sen2cor
#1

I am processing the tiles T32TQS and T33TUM.

I get the following errors several times during the processing run:

Error in xml schema, see report file for details.
Parsing error:
Schema file: S2-PDGS-TAS-DI-PSD-V14.2_Schema\S2_PDI_Level-2A_Datastrip_Metadata.xsd
Details: Element '{http://www.w3.org/2001/XMLSchema}include': Failed to load the document '[..]/Sen2Cor/Lib/site-packages/sen2cor/cfg/S2-PDGS-TAS-DI-PSD-V14.2_Schema/DICO/12/PDGS/dimap/dimap.xsd' for inclusion., line 31

Error in xml schema, see report file for details.
Parsing error:
Schema file: S2-PDGS-TAS-DI-PSD-V14.2_Schema\S2_PDI_Level-1C_Tile_Metadata.xsd
Details: element decl. 'PSI_X', attribute 'type': The QName value '{https://psd-12.sentinel2.eo.esa.int/DICO/12/SY/misc/}A_DOUBLE_WITH_DEG_UNIT_ATTR' does not resolve to a(n) type definition., line 21

Error in xml schema, see report file for details.
Parsing error:
Schema file: S2-PDGS-TAS-DI-PSD-V14.2_Schema\S2_User_Product_Level-2A_Metadata.xsd
Details: Element '{http://www.w3.org/2001/XMLSchema}include': Failed to load the document '[..]/B2_Sentinel-2/Sen2Cor/Lib/site-packages/sen2cor/cfg/S2-PDGS-TAS-DI-PSD-V14.2_Schema/DICO/14/PDGS/dimap/dimap.xsd' for inclusion., line 16

Processing continues and results in Progress[%]: 100.00 : Application terminated successfully.
The resultig files can be read normally.
What do these errors mean and how do I resolve it? Any limitations to the resulting files?

Standalone Installer
OS: Windows 10 & Windows Server 2012

#2

Hi Unic, can you post an example of the L1C product identifier where this mismatches occurr when converting to L2A? I can then check. No, the parsing errors will have no impact on the product itself. It is a quality control of the metadata at the end of the processing which shall verify the xml data against the schemes.

#3

It happend with all tiles that I have observed so far (many are processed without the progress console log due to parallel processing)
e.g: S2B_MSIL1C_20170828T094029_N0205_R036_T34TDR_20170828T094026.SAFE

#4

Hi unnic, I just installed the stand-alone version on a windows 10 machine. Here is the result:

C:\Users\umuellerwilm\AppData\Local\Sen2Cor-2.4.0-win64>L2A_Process.bat …\Testdata\S2A_MSIL1C_20161123T093322_N0204_R136_T34SDJ_20161123T093317.SAFE --resolution=60

Sentinel-2 Level 2A Processor (Sen2Cor), 2.4.0, created: 2017.06.05 started …
selected resolution is 60 m.
Progress[%]: 0.69 : PID-16012, L2A_ProcessTile: processing with resolution 60 m, elapsed time[s]: 1.037
Progress[%]: 0.70 : PID-16012, L2A_ProcessTile: start of pre processing, elapsed time[s]: 0.015
Progress[%]: 0.74 : PID-16012, L2A_Tables: start import, elapsed time[s]: 0.063
Progress[%]: 1.39 : PID-16012, L2A_Tables: band B01 imported, elapsed time[s]: 0.969
Progress[%]: 18.91 : PID-16012, L2A_Tables: band B02 imported, elapsed time[s]: 26.284
Progress[%]: 36.28 : PID-16012, L2A_Tables: band B03 imported, elapsed time[s]: 26.050
Progress[%]: 54.00 : PID-16012, L2A_Tables: band B04 imported, elapsed time[s]: 26.581
Progress[%]: 59.26 : PID-16012, L2A_Tables: band B05 imported, elapsed time[s]: 7.891
Progress[%]: 64.77 : PID-16012, L2A_Tables: band B06 imported, elapsed time[s]: 8.267
Progress[%]: 70.37 : PID-16012, L2A_Tables: band B07 imported, elapsed time[s]: 8.391
Progress[%]: 75.97 : PID-16012, L2A_Tables: band B8A imported, elapsed time[s]: 8.408
Progress[%]: 76.67 : PID-16012, L2A_Tables: band B09 imported, elapsed time[s]: 1.050
Progress[%]: 77.06 : PID-16012, L2A_Tables: band B10 imported, elapsed time[s]: 0.578
Progress[%]: 82.31 : PID-16012, L2A_Tables: band B11 imported, elapsed time[s]: 7.876
Progress[%]: 87.47 : PID-16012, L2A_Tables: band B12 imported, elapsed time[s]: 7.751
Progress[%]: 87.48 : PID-16012, L2A_ProcessTile: start of Scene Classification, elapsed time[s]: 0.010
Progress[%]: 87.61 : PID-16012, Pre process , elapsed time[s]: 0.187
Progress[%]: 88.31 : PID-16012, L2A_SC init , elapsed time[s]: 1.063
Progress[%]: 88.48 : PID-16012, L2A_CSND_1_1 , elapsed time[s]: 0.250
Progress[%]: 88.65 : PID-16012, L2A_CSND_1_2 , elapsed time[s]: 0.250
Progress[%]: 88.77 : PID-16012, L2A_CSND_2_0 , elapsed time[s]: 0.187
Progress[%]: 88.94 : PID-16012, L2A_CSND_2_1 , elapsed time[s]: 0.250
Progress[%]: 89.07 : PID-16012, L2A_CSND_2_1_2, elapsed time[s]: 0.204
Progress[%]: 89.17 : PID-16012, L2A_CSND_2_2 , elapsed time[s]: 0.140
Progress[%]: 89.24 : PID-16012, L2A_CSND_2_3 , elapsed time[s]: 0.110
Progress[%]: 89.39 : PID-16012, L2A_CSND_2_4 , elapsed time[s]: 0.218
Progress[%]: 89.54 : PID-16012, L2A_CSND_2_5 , elapsed time[s]: 0.235
Progress[%]: 89.79 : PID-16012, L2A_CSND_3 , elapsed time[s]: 0.375
Progress[%]: 89.99 : PID-16012, L2A_CSND_5_1 , elapsed time[s]: 0.297
Progress[%]: 90.37 : PID-16012, L2A_CSND_5_2 , elapsed time[s]: 0.562
Progress[%]: 90.63 : PID-16012, L2A_CSND_6 , elapsed time[s]: 0.391
Progress[%]: 90.78 : PID-16012, L2A_CSND_6_2 , elapsed time[s]: 0.234
Progress[%]: 90.94 : PID-16012, L2A_CSND_7 , elapsed time[s]: 0.235
Progress[%]: 93.23 : PID-16012, L2A_SHD , elapsed time[s]: 3.438
Progress[%]: 93.45 : PID-16012, DV recovery , elapsed time[s]: 0.328
Progress[%]: 93.71 : PID-16012, WP recovery , elapsed time[s]: 0.391
Progress[%]: 93.93 : PID-16012, Snow recovery , elapsed time[s]: 0.328
Progress[%]: 94.05 : PID-16012, Soil recovery , elapsed time[s]: 0.187
Progress[%]: 95.09 : PID-16012, Post process , elapsed time[s]: 1.547
Progress[%]: 95.09 : PID-16012, L2A_ProcessTile: start of Atmospheric Correction, elapsed time[s]: 0.000
Progress[%]: 95.22 : PID-16012, L2A_AtmCorr: end of calculation terrain maps, elapsed time[s]: 0.203
Progress[%]: 95.22 : PID-16012, L2A_AtmCorr: start of AOT retrieval at 550nm, elapsed time[s]: 0.000
Progress[%]: 96.17 : PID-16012, L2A_AtmCorr: end of internal classification, elapsed time[s]: 1.422
Progress[%]: 96.43 : PID-16012, L2A_AtmCorr: end of interpolation LUTs, elapsed time[s]: 0.391
Progress[%]: 97.50 : PID-16012, L2A_AtmCorr: end retrieving reference pixels for dark areas, elapsed time[s]: 1.610
Progress[%]: 98.85 : PID-16012, L2A_AtmCorr: end of check for negative reflectance pixels, elapsed time[s]: 2.578
Progress[%]: 99.38 : PID-16012, L2A_AtmCorr: end of visibility index calculation, elapsed time[s]: 1.344
Progress[%]: 99.01 : PID-16012, L2A_AtmCorr: end of AOT retrieval at 550nm, elapsed time[s]: 0.000
Progress[%]: 98.64 : PID-16012, L2A_AtmCorr: start of water vapour retrieval, elapsed time[s]: 0.000
Progress[%]: 98.72 : PID-16012, L2A_AtmCorr: end of water vapour retrieval preparation, elapsed time[s]: 0.688
Progress[%]: 99.65 : PID-16012, L2A_AtmCorr: end of water vapour retrieval, elapsed time[s]: 10.673
Progress[%]: 99.28 : PID-16012, L2A_AtmCorr: preparation of surface reflectance retrieval, elapsed time[s]: 0.000
Progress[%]: 99.63 : PID-16012, L2A_AtmCorr: end of surface reflectance retrieval preparation, elapsed time[s]: 1.781
Progress[%]: 99.67 : PID-16012, L2A_AtmCorr: end of surface reflectance retrieval, elapsed time[s]: 17.799
Progress[%]: 99.65 : PID-16012, L2A_AtmCorr: end of rho retrieval step 2, elapsed time[s]: 6.688
Progress[%]: 99.29 : PID-16012, L2A_ProcessTile: start of post processing, elapsed time[s]: 0.016
Progress[%]: 98.92 : PID-16012, L2A_Tables: start export, elapsed time[s]: 0.000
Progress[%]: 99.22 : PID-16012, L2A_Tables: band B01 exported, elapsed time[s]: 1.000
Progress[%]: 99.53 : PID-16012, L2A_Tables: band B02 exported, elapsed time[s]: 1.031
Progress[%]: 99.63 : PID-16012, L2A_Tables: band B03 exported, elapsed time[s]: 1.016
Progress[%]: 99.63 : PID-16012, L2A_Tables: band B04 exported, elapsed time[s]: 1.016
Progress[%]: 99.63 : PID-16012, L2A_Tables: band B05 exported, elapsed time[s]: 1.062
Progress[%]: 99.63 : PID-16012, L2A_Tables: band B06 exported, elapsed time[s]: 1.094
Progress[%]: 99.63 : PID-16012, L2A_Tables: band B07 exported, elapsed time[s]: 1.112
Progress[%]: 99.63 : PID-16012, L2A_Tables: band B8A exported, elapsed time[s]: 1.125
Progress[%]: 99.63 : PID-16012, L2A_Tables: band B09 exported, elapsed time[s]: 0.907
Progress[%]: 99.63 : PID-16012, L2A_Tables: band B11 exported, elapsed time[s]: 1.046
Progress[%]: 99.63 : PID-16012, L2A_Tables: band B12 exported, elapsed time[s]: 1.038
Progress[%]: 99.40 : PID-16012, L2A_Tables: band SCL exported, elapsed time[s]: 0.203
Progress[%]: 99.08 : PID-16012, L2A_Tables: band SNW exported, elapsed time[s]: 0.078
Progress[%]: 98.83 : PID-16012, L2A_Tables: band CLD exported, elapsed time[s]: 0.172
Progress[%]: 98.55 : PID-16012, L2A_Tables: band AOT exported, elapsed time[s]: 0.141
Progress[%]: 98.62 : PID-16012, L2A_Tables: band WVP exported, elapsed time[s]: 0.672
Progress[%]: 98.50 : PID-16012, L2A_Tables: band PVI exported, elapsed time[s]: 0.375
Progress[%]: 99.23 : PID-16012, L2A_Tables: band TCI exported, elapsed time[s]: 1.641
Progress[%]: 98.87 : PID-16012, L2A_Tables: stop export, elapsed time[s]: 0.015
Progress[%]: 100.00 : Application terminated successfully.

C:\Users\umuellerwilm\AppData\Local\Sen2Cor-2.4.0-win64>L2A_Process --help
usage: L2A_Process.py [-h] [–resolution {10,20,60}] [–sc_only] [–cr_only]
[–refresh] [–GIP_L2A GIP_L2A]
[–GIP_L2A_SC GIP_L2A_SC] [–GIP_L2A_AC GIP_L2A_AC]
directory

Sentinel-2 Level 2A Processor (Sen2Cor). Version: 2.4.0, created: 2017.06.05,
supporting Level-1C product version: 14.

positional arguments:
directory Directory where the Level-1C input files are located

optional arguments:
-h, --help show this help message and exit
–resolution {10,20,60}
Target resolution, can be 10, 20 or 60m. If omitted,
all resolutions will be processed
–sc_only Performs only the scene classification at 60 or 20m
resolution
–cr_only Performs only the creation of the L2A product tree, no
processing
–refresh Performs a refresh of the persistent configuration
before start
–GIP_L2A GIP_L2A Select the user GIPP
–GIP_L2A_SC GIP_L2A_SC
Select the scene classification GIPP
–GIP_L2A_AC GIP_L2A_AC
Select the atmospheric correction GIPP

Are you really shure to use the stand alone Processor? What is the output if you call L2A_Process–help ?

#5

Yes, it’s the standalone installer

usage: L2A_Process.py [-h] [--resolution {10,20,60}] [--sc_only] [--cr_only]
                      [--refresh] [--GIP_L2A GIP_L2A]
                      [--GIP_L2A_SC GIP_L2A_SC] [--GIP_L2A_AC GIP_L2A_AC]
                      directory

Sentinel-2 Level 2A Processor (Sen2Cor). Version: 2.4.0, created: 2017.06.05,
supporting Level-1C product version: 14.

positional arguments:
  directory             Directory where the Level-1C input files are located

optional arguments:
  -h, --help            show this help message and exit
  --resolution {10,20,60}
                        Target resolution, can be 10, 20 or 60m. If omitted,
                        all resolutions will be processed
  --sc_only             Performs only the scene classification at 60 or 20m
                        resolution
  --cr_only             Performs only the creation of the L2A product tree, no
                        processing
  --refresh             Performs a refresh of the persistent configuration
                        before start
  --GIP_L2A GIP_L2A     Select the user GIPP
  --GIP_L2A_SC GIP_L2A_SC
                        Select the scene classification GIPP
  --GIP_L2A_AC GIP_L2A_AC
                        Select the atmospheric correction GIPP
#6

bumping @umwilm

#7

hi @unnic,

I encountered a similar error in v2.5.5 today.
Have you modified the stand-alone installer, e.g. copied it to a different location? Maybe some files were not extracted properly?

A bit of context: I’m currently isolating the sen2cor code and setting up a setup.py installer for it.
I managed to fix my parsing error by making sure that all files from Sen2Cor-02.05.05-$OS/lib/python2.7/site-packages/sen2cor/cfg/** are correctly copied to the new location of my script (venv).
When building a setup.py one can use the data_files parameter to specify additional files.

Edit: Sorry to necro-bump this thread. As I did not find a solution to this somewhere else, I though it might be useful to some of you folks anyway.