Sen2cor stalls due to missing atm. lookup file

Optical Toolbox sen2cor
#1

Hi everyone,

It seems like the sen2cor estimated water vapour content can exceed the range of the included atmospheric lookup tables, in which case the corresponding LUT is not found and also not interpolated (presumably due to a missing lower bound).
If this happens L2A_Process is stuck in waiting and does not terminate.

This happens for example with the following scene:
S2A_OPER_PRD_MSIL1C_PDMC_20161007T190917_R108_V20150829T103026_20150829T103028

####The gipp file:
L2A_GIPP.xml (4.1 KB)

The call:

L2A_Process --resolution=20 S2A_OPER_PRD_MSIL1C_PDMC_20161007T190917_R108_V20150829T103026_20150829T103028.SAFE

The problem:

It is looking for h99000_wv02_rura.atm but the closest available would be h99000_wv04_rura.atm.

...
Progress[%]: 41.72 : PID-7245, L2A_AtmCorr: end of AOT retrieval at 550nm, elapsed time[s]: 0.887
Process L2A_ProcessTile-1:
Traceback (most recent call last):
  File "/opt/anaconda2/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/opt/anaconda2/lib/python2.7/site-packages/sen2cor-2.2.1-py2.7.egg/sen2cor/L2A_ProcessTile.py", line 132, in  run
    if self.process_20() == False:
  File "/opt/anaconda2/lib/python2.7/site-packages/sen2cor-2.2.1-py2.7.egg/sen2cor/L2A_ProcessTile.py", line 162, in process_20
    return self.process()
  File "/opt/anaconda2/lib/python2.7/site-packages/sen2cor-2.2.1-py2.7.egg/sen2cor/L2A_ProcessTile.py", line 219, in process
    if(ac.automaticAerosolDetection() == False):
  File "L2A_AtmCorr.py", line 3292, in L2A_AtmCorr.L2A_AtmCorr.automaticAerosolDetection (L2A_AtmCorr.c:67233)
  File "L2A_AtmCorr.py", line 3329, in L2A_AtmCorr.L2A_AtmCorr.calcDratioAerosol (L2A_AtmCorr.c:68156)
  File "L2A_AtmCorr.py", line 3156, in L2A_AtmCorr.L2A_AtmCorr.process (L2A_AtmCorr.c:64774)
  File "L2A_AtmCorr.py", line 5672, in L2A_AtmCorr.L2A_AtmCorr.masking_a3 (L2A_AtmCorr.c:126563)
  File "L2A_AtmCorr.py", line 6198, in L2A_AtmCorr.L2A_AtmCorr.masking_a3_skip1 (L2A_AtmCorr.c:138367)
  File "L2A_AtmCorr.py", line 9717, in L2A_AtmCorr.L2A_AtmCorr.apda1_lut_constvis_a3 (L2A_AtmCorr.c:221649)
  File "L2A_AtmCorr.py", line 4406, in L2A_AtmCorr.L2A_AtmCorr.altit3_atm (L2A_AtmCorr.c:97015)
  File "L2A_AtmCorr.py", line 4018, in L2A_AtmCorr.L2A_AtmCorr.read_atm_hyper_a3 (L2A_AtmCorr.c:84826)
  File "/opt/anaconda2/lib/python2.7/genericpath.py", line 57, in getsize
    return os.stat(filename).st_size
OSError: [Errno 2] No such file or directory: '/opt/anaconda2/lib/python2.7/site-packages/sen2cor-2.2.1-py2.7.egg/sen2cor/lib/20_60/h99000_wv02_rura.atm'

Sen2cor version: 2.2.1
This forum is the closest to a bug tracker I could find. If there’s an official tracker, please let me know

Error using sen2cor v 2.3.0 with LUT parameter set to 'AUTO'
#2

Don’t set Mid Latitude and Aerosol Type in the GIP file to ‘AUTO’, its not working.

Cheers,
Ognjen

#3

Thanks for this info Ognjen. Now I also found your earlier report on the same issue.

I just wish there was an official bug tracker with known issues instead of a forum mixing support and bugs. But that’s another story.

#4

I ran a few more tests on this particular scene and found that only the Mid_Latitude estimation is broken.
The aerosol type estimation works for all three options (RURAL, MARITIME, AUTO) as long as Mid_latitude is not set to AUTO.
I hope this helps to corner the bug (@umwilm)

#5

I confirm that there is a problem with the automatic aerosol detection. The issue has already been fixed and will be part of the next sen2cor delivery which is coming soon. The official release will be announced on the ESA web page.
Cheers, umwilm

#6

Hi all,
I have installed SNAP and sen2cor 2.2.3 on Ubuntu 16.04 (server), and i have already the same problem: sen2cor seems to search a file called: “h99000_wv02_rura.atm” that doesn’t exist…
i tried to change the Mid-latitude and the aerosol type like you said but it doesn’t fix the problem…
Any suggestions ?
Any news about a 2017 release of sen2cor ?
Thanks (Happy new year everybody)

#7

The most recent version of sen2cor is 2.3.0, but I guess that’s what you meant?
Nevertheless, I had the same error also occur to me with the 2.3.0 version, but for me fixing mid_latitude to either SUMMER or WINTER did made it work again (with Ozone=0, i.e. from metadata)
I’m not happy with this solution, but at least it runs and the values do look ok.

#8

Hi Benjamin,
I made a mistake, i have already upgrade my sen2cor version (now 2.3.0).
I tried to change settings but i had the same error

[Errno 2] No such file or directory: ‘/home/geouser/anaconda2/lib/python2.7/site-packages/sen2cor-2.3.0-py2.7.egg/sen2cor/lib/20_60/h99000_wv02_rura.atm’

Do you think you can upload your L2A_GIP.xml file ? I would like to compare with my settings…
Thank you very much.

#9

sure: L2A_GIPP.xml (4.5 KB)

#10

Thank you for your help
it’s ok, sen2cor is functionnal
i had a problem with my python interpreter…
Thank you very much

#11

Hello All,

unfortunatelly problem with this error No such file or directory: ‘/home/geouser/anaconda2/lib/python2.7/site-packages/sen2cor-2.3.0-py2.7.egg/sen2cor/lib/20_60/h99000_wv02_rura.atm’ is no solved in last sen2cor version (2.3.1). If i set in L2A_GIPP file Aerosol type Winter i get this error again. When i set aerosol type Summer everything is ok but my images are from January or February and Summer aerosol type is not relevant.

Regards