Sen2cor 2.5.5 : AttributeError: no such child: ESACCI_WaterBodies_Map


#1

Hi everyone,
I installed Sen2cor 2.5.5 with the standalone installer and ecverything went very smoothly.
I then sourced the new L2A_Bashrc file
source /mnt/data/home/hagolleo/PROG/SEN2COR/Sen2Cor-02.05.05-Linux64/L2A_Bashrc

But Sen2cor crashes with

Traceback (most recent call last):
File “/mnt/data/home/hagolleo/PROG/SEN2COR/Sen2Cor-02.05.05-Linux64/lib/python2.7/multiprocessing/process.py”, line 258, in _bootstrap
self.run()
File “/mnt/data/home/hagolleo/PROG/SEN2COR/Sen2Cor-02.05.05-Linux64/lib/python2.7/site-packages/sen2cor/L2A_ProcessTile.py”, line 136, in run
if not self.process_10():
File “/mnt/data/home/hagolleo/PROG/SEN2COR/Sen2Cor-02.05.05-Linux64/lib/python2.7/site-packages/sen2cor/L2A_ProcessTile.py”, line 169, in process_10
self.config.readPreferences()
File “/mnt/data/home/hagolleo/PROG/SEN2COR/Sen2Cor-02.05.05-Linux64/lib/python2.7/site-packages/sen2cor/L2A_Config.py”, line 3762, in readPreferences
par = node.ESACCI_WaterBodies_Map
File “src/lxml/lxml.objectify.pyx”, line 229, in lxml.objectify.ObjectifiedElement.getattr (src/lxml/lxml.objectify.c:3837)
File “src/lxml/lxml.objectify.pyx”, line 450, in lxml.objectify._lookupChildOrRaise (src/lxml/lxml.objectify.c:6541)
AttributeError: no such child: ESACCI_WaterBodies_Map

Do I need to download an ESA CCI file ?
thanks in advance
Olivier


#2

I get the exact same error… Hoping that somebody can help us out!


#3

I seem to have solved it for now. I spend hours to identify the error in the respective python file, but everything seems to be ok. I also checked the relevant xml file, also no issues found here. For some reason the XML parser seem to make an error out of the blue; it simply does not read those nodes from the xml file, but I just can’t figure out why? It’s a very simple read! It reads all other nodes just fine. Anyways I reinstalled sen2cor and I deleted the 2A folder corresponding to the 1C folder that I had been trying to process. And now it runs… A mystery to me! Hoping that somebody can solve it and enlighten us!


#4

Hallo Olivier +++,

  1. Sen2Cor should be running without CCI-files. I will check later today if I can reproduce your problem reported.
  2. You need to download an ESA CCI file if you like to use the recent advantages of Sen2_Cor scene classification. Open http://maps.elie.ucl.ac.be/CCI/viewer/download.php , there you find an option “February 2018 - A LC package prepared for Sen2Cor users” for download including instructions how to activate it.
    Unfortunatley I don’t found any remark for this in the Sen2Cor documentation provided (software releaso note SRN and user manual SUM). Sorry, we have to change this.
    Thank you for reporting this problem.

#5

Thank you Greforb,

You’re right, I did not reinstall but removed the existing L2A files, and it worked… until it reached the
‘L2A_Tables’ object has no attribute ‘_L2A_Tile_PVI_File’ error.

And thank you Bringried, I’ll try to download the CCI land cover map. If I undestoof well, Sen2cor cloud detection threshold are now dependent on the land cover class ?

Bringfried, any idea on the issue with ‘_L2A_Tile_PVI_File’ ?

Best regards,
Olivier


#6

Olivier, the S2-PDGS-MPC-L2A-SUM-V2.5.5_V2.pdf on the Sen2Cor download page was now updated to give the information how to download the CCI-files.
The issue with the PVI-file seems to be connected with the new PDGS-mode included in Sen2Cor 2.5. There are many posts related to it. I gave few more details in my reply to post Sen2cor 2.5.5 - No such file _PVI.jp2

Best regards
Bringfried


#7

Hi Bringfried,

Removing --resolution 10 in the command line works and seems to remove the “PVI issue”.

Regarding CCI data base, I downloaded the CCI as specified in the SUM and unzipped it, and then moved the CCI4SEN2COR (with tar files) in $SEN2COR_BIN/aux_data/CCI4SEN2COR . I also tried to untar the files. In each case, I got the following warnings:

[ERROR]  L2A_Tables: 1427   ESA CCI Water Bodies map not present, water detection will be performed without a priori information
[ERROR]  L2A_Tables: 1427   ESA CCI Water Bodies map not present, water detection will be performed without a priori information
[ERROR]  L2A_Tables: 1481   ESA CCI Land Cover map not present, cloud detection over urban areas will be performed without a priori information
[ERROR]  L2A_Tables: 1481   ESA CCI Land Cover map not present, cloud detection over urban areas will be performed without a priori information
[ERROR]  L2A_Tables: 1558   ESA CCI Snow Condition map not present, no snow map post-processing will be done
[ERROR]  L2A_Tables: 1558   ESA CCI Snow Condition map not present, no snow map post-processing will be done

Finally, I tried to just copy the files in $SEN2COR_BIN/aux_data/, without CCI4SEN2COR, and it worked. Maybe adding a more accurate description in the SUM would be helpful.

Anyway, thanks a lot for your help !
Best regards,
Olivier


#8

I am having the same issue but unable to get it working without the warnings you cited above @OHagolle. Can you share the layout of files you see if you do ls -lRa $SEN2COR_BIN/aux_data/ ?

Are there any configuration changes you need to specify?

I’d much appreciate it!


#9

You can also specify your own L2A_CAL_SC_GIPP.xml file to customize the location. Looking at the original xml file used by sen2cor located in

sen2cor/lib/python2.7/site-packages/sen2cor/cfg/L2A_CAL_SC_GIPP.xml

  <Scene_Classification>
    <References>
      <Snow_Map>GlobalSnowMap.tiff</Snow_Map>
      <ESACCI_WaterBodies_Map>ESACCI-LC-L4-WB-Map-150m-P13Y-2000-v4.0.tif</ESACCI_WaterBodies_Map>
      <ESACCI_LandCover_Map>ESACCI-LC-L4-LCCS-Map-300m-P1Y-2015-v2.0.7.tif</ESACCI_LandCover_Map>
      <ESACCI_SnowCondition_Map_Dir>ESACCI-LC-L4-Snow-Cond-500m-P13Y7D-2000-2012-v2.0</ESACCI_SnowCondition_Map_Dir>

The files have to be in the same folder as GlobalSnowMap.tiff

I hope that helps.


#10

I had misinterpreted the instructions in the SUM and I’d interpreted $SEN2COR_BIN as Sen2Cor-02.05.05-Darwin64/bin/ but that should really be Sen2Cor-02.05.05-Darwin64/lib/python2.7/site-packages/sen2cor/

@OHagolle your tip to ensure the unzip and untar operations match the internal config file did it for me! Thank you very much!

Now I see log messages acknowledging the use of land cover products.

Progress[%]: 4.47 : PID-55023, L2A_ProcessTile: start of Scene Classification, elapsed time[s]: 0.003, total: 0:02:20.290732
Progress[%]: 4.47 : PID-55023, Pre process   , elapsed time[s]: 0.107, total: 0:02:20.397994
Progress[%]: 4.49 : PID-55023, L2A_SC init   , elapsed time[s]: 0.440, total: 0:02:20.837940
Progress[%]: 4.49 : PID-55023, L2A_CSND_1_1  , elapsed time[s]: 0.131, total: 0:02:20.968687
Progress[%]: 4.49 : PID-55023, L2A_CSND_1_2  , elapsed time[s]: 0.140, total: 0:02:21.109073
Progress[%]: 4.50 : PID-55023, L2A_CSND_2_0  , elapsed time[s]: 0.225, total: 0:02:21.333916
Progress[%]: 4.51 : PID-55023, L2A_CSND_2_1  , elapsed time[s]: 0.147, total: 0:02:21.481201
Progress[%]: 4.51 : PID-55023, L2A_CSND_2_1_2, elapsed time[s]: 0.099, total: 0:02:21.579742
Progress[%]: 4.51 : PID-55023, L2A_CSND_2_2  , elapsed time[s]: 0.060, total: 0:02:21.640174
Progress[%]: 4.51 : PID-55023, L2A_CSND_2_3  , elapsed time[s]: 0.061, total: 0:02:21.701133
Progress[%]: 4.52 : PID-55023, L2A_CSND_2_4  , elapsed time[s]: 0.108, total: 0:02:21.808725
Progress[%]: 4.52 : PID-55023, L2A_CSND_2_5  , elapsed time[s]: 0.175, total: 0:02:21.983444
Progress[%]: 4.52 : PID-55023, L2A_SnowPostProcessingCCI  , elapsed time[s]: 0.036, total: 0:02:22.019871
Progress[%]: 4.53 : PID-55023, L2A_CSND_3    , elapsed time[s]: 0.203, total: 0:02:22.222884
Progress[%]: 4.54 : PID-55023, L2A_CSND_5_1  , elapsed time[s]: 0.166, total: 0:02:22.388934
Progress[%]: 4.55 : PID-55023, L2A_CSND_5_2  , elapsed time[s]: 0.315, total: 0:02:22.703697
Progress[%]: 4.55 : PID-55023, L2A_CSND_6    , elapsed time[s]: 0.206, total: 0:02:22.909794
Progress[%]: 4.56 : PID-55023, L2A_CSND_6_2  , elapsed time[s]: 0.125, total: 0:02:23.035237
Progress[%]: 4.56 : PID-55023, L2A_CSND_7    , elapsed time[s]: 0.150, total: 0:02:23.185263
Progress[%]: 4.68 : PID-55023, L2A_SHD       , elapsed time[s]: 3.811, total: 0:02:26.996523
Progress[%]: 4.69 : PID-55023, DV recovery   , elapsed time[s]: 0.199, total: 0:02:27.195501
Progress[%]: 4.69 : PID-55023, WP recovery   , elapsed time[s]: 0.190, total: 0:02:27.385289
Progress[%]: 4.70 : PID-55023, WP recovery with CCI Water Bodies at 150m  , elapsed time[s]: 0.068, total: 0:02:27.453522
Progress[%]: 4.70 : PID-55023, Water Pixels cleaning with DEM, elapsed time[s]: 0.070, total: 0:02:27.523483
Progress[%]: 4.70 : PID-55023, Cloud Shadow Pixels cleaning with DEM, elapsed time[s]: 0.026, total: 0:02:27.549430
Progress[%]: 4.71 : PID-55023, Topographic shadows classification over snow in mountainous area with DEM, elapsed time[s]: 0.179, total: 0:02:27.728234
Progress[%]: 4.71 : PID-55023, Snow recovery , elapsed time[s]: 0.202, total: 0:02:27.930109
Progress[%]: 4.72 : PID-55023, Soil recovery , elapsed time[s]: 0.101, total: 0:02:28.031343
Progress[%]: 4.72 : PID-55023, Land recovery with B10, B09 and B8A , elapsed time[s]: 0.224, total: 0:02:28.255574
Progress[%]: 4.73 : PID-55023, Cirrus recovery with B10 , elapsed time[s]: 0.136, total: 0:02:28.391253
Progress[%]: 4.75 : PID-55023, Urban and Bare pixel recovery with CCI Land Cover Map at 300 m , elapsed time[s]: 0.743, total: 0:02:29.134600
Progress[%]: 4.76 : PID-55023, Post process  , elapsed time[s]: 0.410, total: 0:02:29.544969
Progress[%]: 4.76 : PID-55023, L2A_ProcessTile: start of Automatic Aerosol Type Detection, elapsed time[s]: 0.001, total: 0:02:29.546316
Progress[%]: 4.76 : PID-55023, L2A_AtmCorr: testing mid latitude: SUMMER, aerosol type: RURAL, elapsed time[s]: 0.003, total: 0:02:29.548938
Progress[%]: 4.82 : PID-55023, L2A_AtmCorr: end of calculation terrain maps, elapsed time[s]: 1.908, total: 0:02:31.457391
Progress[%]: 4.91 : PID-55023, L2A_AtmCorr: start of AOT retrieval at 550nm, elapsed time[s]: 2.547, total: 0:02:34.003954
Progress[%]: 5.16 : PID-55023, L2A_AtmCorr: end of internal classification, elapsed time[s]: 7.983, total: 0:02:41.986947
Progress[%]: 5.18 : PID-55023, L2A_AtmCorr: end of interpolation LUTs, elapsed time[s]: 0.596, total: 0:02:42.582927