Sen2cor hangs while doing scene classification

Optical Toolbox sen2cor
#1

Running sen2cor (2.3.1) under Debian with command line:

L2A_Process --resolution 10 /sentinel_products_unzip/S2A_MSIL1C_20170602T090021_N0205_R007_T36VWJ_20170602T090020.SAFE/

I got the output:

Sentinel-2 Level 2A Processor (Sen2Cor), 2.3.1, created: 2017.02.03 started ...
Syntax error in metadata, see report file for details.
Parsing error:
Schema file: L2A_CAL_AC_GIPP.xsd
Details: Element 'wavelength', attribute 'c0': '' is not a valid value of the local atomic type. (line 0)
Syntax error in metadata, see report file for details.
Parsing error:
Schema file: L2A_GIPP.xsd
Details: Element 'PSD_Scheme', attribute 'PSD_Version': [facet 'length'] The value '' has a length of '0'; this differs from the allowed length of '2'. (line 0)
selected resolution is 10 m.
...
<more logs ommited>
...
Progress[%]: 0.04 : PID-52, L2A_Tables: start import, elapsed time[s]: 0.116
Progress[%]: 0.23 : PID-52, L2A_Tables: band B01 imported, elapsed time[s]: 4.657
Progress[%]: 1.48 : PID-52, L2A_Tables: band B02 imported, elapsed time[s]: 29.852
Progress[%]: 2.74 : PID-52, L2A_Tables: band B03 imported, elapsed time[s]: 30.429
Progress[%]: 4.05 : PID-52, L2A_Tables: band B04 imported, elapsed time[s]: 31.410
Progress[%]: 4.43 : PID-52, L2A_Tables: band B05 imported, elapsed time[s]: 9.048
Progress[%]: 4.81 : PID-52, L2A_Tables: band B06 imported, elapsed time[s]: 9.203
Progress[%]: 5.25 : PID-52, L2A_Tables: band B07 imported, elapsed time[s]: 10.391
Progress[%]: 5.70 : PID-52, L2A_Tables: band B8A imported, elapsed time[s]: 10.897
Progress[%]: 5.89 : PID-52, L2A_Tables: band B09 imported, elapsed time[s]: 4.473
Progress[%]: 6.06 : PID-52, L2A_Tables: band B10 imported, elapsed time[s]: 4.257
Progress[%]: 6.54 : PID-52, L2A_Tables: band B11 imported, elapsed time[s]: 11.357
Progress[%]: 6.91 : PID-52, L2A_Tables: band B12 imported, elapsed time[s]: 8.987
Progress[%]: 6.91 : PID-52, L2A_ProcessTile: start of Scene Classification, elapsed time[s]: 0.019
Progress[%]: 6.94 : PID-52, Pre process   , elapsed time[s]: 0.778
Progress[%]: 6.98 : PID-52, L2A_SC init   , elapsed time[s]: 0.807
Progress[%]: 7.03 : PID-52, L2A_CSND_1_1  , elapsed time[s]: 1.357
Progress[%]: 7.08 : PID-52, L2A_CSND_1_2  , elapsed time[s]: 1.196
Progress[%]: 7.10 : PID-52, L2A_CSND_2_0  , elapsed time[s]: 0.278
Progress[%]: 7.14 : PID-52, L2A_CSND_2_1  , elapsed time[s]: 1.108
Progress[%]: 7.18 : PID-52, L2A_CSND_2_1_2, elapsed time[s]: 0.970
Progress[%]: 7.21 : PID-52, L2A_CSND_2_2  , elapsed time[s]: 0.563
Progress[%]: 7.23 : PID-52, L2A_CSND_2_3  , elapsed time[s]: 0.552
Progress[%]: 7.27 : PID-52, L2A_CSND_2_4  , elapsed time[s]: 0.987
Progress[%]: 7.32 : PID-52, L2A_CSND_2_5  , elapsed time[s]: 1.163
Progress[%]: 7.39 : PID-52, L2A_CSND_3    , elapsed time[s]: 1.727
Progress[%]: 7.45 : PID-52, L2A_CSND_5_1  , elapsed time[s]: 1.460
Progress[%]: 7.56 : PID-52, L2A_CSND_5_2  , elapsed time[s]: 2.588
Progress[%]: 7.63 : PID-52, L2A_CSND_6    , elapsed time[s]: 1.782
Progress[%]: 7.68 : PID-52, L2A_CSND_6_2  , elapsed time[s]: 1.216
Progress[%]: 7.74 : PID-52, L2A_CSND_7    , elapsed time[s]: 1.403

and thats all, process hangs, and does nothing

top - 07:20:00 up 21:06,  0 users,  load average: 0.07, 0.08, 0.19
Tasks:   5 total,   1 running,   4 sleeping,   0 stopped,   0 zombie
%Cpu(s):  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem:   4023148 total,   265832 used,  3757316 free,     5188 buffers
KiB Swap:  1048572 total,   456144 used,   592428 free.    36128 cached Mem

   PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
     1 root      20   0    4220      0      0 S   0.0  0.0   0:00.06 tini
     5 root      20   0   21952     76      0 S   0.0  0.0   0:00.02 bash
    38 root      20   0  899456   2052   1344 S   0.0  0.1   0:01.73 L2A_Proce+
    54 root      20   0   21944   3484   2984 S   0.0  0.1   0:00.02 bash
    58 root      20   0   23616   2708   2372 R   0.0  0.1   0:00.00 top

googling lead me to this topic, but it doesn’t halped me.

What should i do? how i can fix this or by pass?

#2

Here is stacktrace for ctrl+c interruption:

Traceback (most recent call last):
  File "/opt/conda/bin/L2A_Process", line 11, in <module>
    load_entry_point('sen2cor==2.3.1', 'console_scripts', 'L2A_Process')()
  File "/opt/conda/lib/python2.7/site-packages/sen2cor-2.3.1-py2.7.egg/sen2cor/L2A_Process.py", line 305, in main
    result = scheduler.sync()
  File "/opt/conda/lib/python2.7/site-packages/sen2cor-2.3.1-py2.7.egg/sen2cor/L2A_Schedule.py", line 71, in sync
    q = self._msgQueue.get()
  File "/opt/conda/lib/python2.7/multiprocessing/queues.py", line 117, in get
    res = self._recv()
KeyboardInterrupt

If need more info about system, i can give it

#3

For test purposes, i increased RAM size for linux from 4GB to 8GB, and thats solved my problem. No more hangs, and process keep going.

Thanks for everybody who helped me with this tool )

May be sen2cor should write warning massage about RAM size? or fall with some error with message about memory allocation problems?

#4

Hi all!

I have the same problem, but from an earlier part of the classification process

The issue I have is in the latest Sen2Cor 2.4.0 correction of older imagery with a naming convention such as:

S2A_OPER_PRD_MSIL1C_PDMC_20161016T055811_R006_V20161015T071852_20161015T073107

Thieprocessing that appears is as follows:

image.png645x583 12.9 KB
and it just stalls at this, and has been like this for 24 hours.
I have tried it on images with the newer naming convention e.g.:

S2A_MSIL1C_20161214T072312_N0204_R006_T37MGT_20161214T073653

and it works - so it may be an issue with the earlier imagery.
However, this older imagery worked with Sen2Cor 2.3.0

I have been using both the standalone and Anaconda software.

I have tried to increase the ram, but this had no effect unfortunately.

Any help you can give would be greatly appreciated,

Best wishes,
Will