IndexError in L2A_AtmCorr.L2A_AtmCorr.ref_pixel_wfov


#1

When performing L2A_Process --resolution 10 on product 0e601fc1-4e84-45de-94dc-e8980934751d with Sen2Cor version 2.0.6, I get the following error:

Traceback (most recent call last):
  File "C:\Anaconda3\envs\python_2_7\Scripts\L2A_Process-script.py", line 9, in <module>
    load_entry_point('sen2cor==2.0.6', 'console_scripts', 'L2A_Process')()
  File "C:\Anaconda3\envs\python_2_7\lib\site-packages\sen2cor-2.0.6-py2.7.egg\sen2cor\L2A_Process.py", line 283, in main
    result = processor.selectAndProcess(tile)
  File "C:\Anaconda3\envs\python_2_7\lib\site-packages\sen2cor-2.0.6-py2.7.egg\sen2cor\L2A_Process.py", line 83, in selectAndProcess
    self.selectAndProcess(tile)
  File "C:\Anaconda3\envs\python_2_7\lib\site-packages\sen2cor-2.0.6-py2.7.egg\sen2cor\L2A_Process.py", line 107, in selectAndProcess
    self._processed20 = self.process()
  File "C:\Anaconda3\envs\python_2_7\lib\site-packages\sen2cor-2.0.6-py2.7.egg\sen2cor\L2A_Process.py", line 148, in process
    if(ac.process() == False):
  File "sen2cor\L2A_AtmCorr.py", line 1379, in sen2cor.L2A_AtmCorr.L2A_AtmCorr.process (L2A_AtmCorr.c:29184)
  File "sen2cor\L2A_AtmCorr.py", line 4822, in sen2cor.L2A_AtmCorr.L2A_AtmCorr.ref_pixel_wfov (L2A_AtmCorr.c:110875)
IndexError: index 915 is out of bounds for axis 1 with size 1

Any suggestions?


#2

I noticed a new/beta version of sen2cor on the ftp (2.2.0). I set up a batch job yesterday, don’t know if it has worked yet and cant comment on the results but 4 threads seems to work. I changed the startup script so I do not have a polluted path. Less issues getting a job running than 2.0.6.


#3

I have seen this error today on product a298bd58-5fa4-40a0-bf52-535904027a67

File "L2A_AtmCorr.py", line 6696, in L2A_AtmCorr.L2A_AtmCorr.ref_pixel_wfov (L2A_AtmCorr.c:150525) IndexError: index 1 is out of bounds for axis 1 with size 1

Any ideas on solution?

v2.2.1


#4

I have a similar error with two of the tiles I want to process:

Progress[%]: 22.22 : PID-21196, L2A_AtmCorr: end of interpolation LUTs, elapsed time[s]: 0.683
Process L2A_ProcessTile-1:
Traceback (most recent call last):
  File "/home/gert/anaconda2/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/gert/anaconda2/lib/python2.7/site-packages/sen2cor-2.3.0-py2.7.egg/sen2cor/L2A_ProcessTile.py", line 135, in run
    if self.process_10() == False:
  File "/home/gert/anaconda2/lib/python2.7/site-packages/sen2cor-2.3.0-py2.7.egg/sen2cor/L2A_ProcessTile.py", line 178, in process_10
    if not self.process_20():
  File "/home/gert/anaconda2/lib/python2.7/site-packages/sen2cor-2.3.0-py2.7.egg/sen2cor/L2A_ProcessTile.py", line 157, in process_20
    return self.process()
  File "/home/gert/anaconda2/lib/python2.7/site-packages/sen2cor-2.3.0-py2.7.egg/sen2cor/L2A_ProcessTile.py", line 223, in process
    if(ac.process() == False):
  File "sen2cor/L2A_AtmCorr.py", line 3177, in sen2cor.L2A_AtmCorr.L2A_AtmCorr.process (L2A_AtmCorr.c:64786)
  File "sen2cor/L2A_AtmCorr.py", line 6742, in sen2cor.L2A_AtmCorr.L2A_AtmCorr.ref_pixel_wfov (L2A_AtmCorr.c:150183)
IndexError: index 2241 is out of bounds for axis 1 with size 1

And:

Progress[%]: 24.94 : PID-21254, L2A_AtmCorr: end of interpolation LUTs, elapsed time[s]: 0.651
Process L2A_ProcessTile-1:
Traceback (most recent call last):
  File "/home/gert/anaconda2/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/gert/anaconda2/lib/python2.7/site-packages/sen2cor-2.3.0-py2.7.egg/sen2cor/L2A_ProcessTile.py", line 135, in run
    if self.process_10() == False:
  File "/home/gert/anaconda2/lib/python2.7/site-packages/sen2cor-2.3.0-py2.7.egg/sen2cor/L2A_ProcessTile.py", line 178, in process_10
    if not self.process_20():
  File "/home/gert/anaconda2/lib/python2.7/site-packages/sen2cor-2.3.0-py2.7.egg/sen2cor/L2A_ProcessTile.py", line 157, in process_20
    return self.process()
  File "/home/gert/anaconda2/lib/python2.7/site-packages/sen2cor-2.3.0-py2.7.egg/sen2cor/L2A_ProcessTile.py", line 223, in process
    if(ac.process() == False):
  File "sen2cor/L2A_AtmCorr.py", line 3177, in sen2cor.L2A_AtmCorr.L2A_AtmCorr.process (L2A_AtmCorr.c:64786)
  File "sen2cor/L2A_AtmCorr.py", line 6742, in sen2cor.L2A_AtmCorr.L2A_AtmCorr.ref_pixel_wfov (L2A_AtmCorr.c:150183)
IndexError: index 274031 is out of bounds for axis 1 with size 1

I am processing tile 31UFS of date 10th of June 2016 an 31UFS of 11th of May 2016


#5

The error still persists in sen2cor 2.3.0
Scene: S2A_OPER_PRD_MSIL1C_PDMC_20170101T013615_R050_V20150924T090726_20150924T090726.SAFE
Granule: S2A_OPER_MSI_L1C_TL_EPA__20161231T221649_A001334_T36UUG_N02.04

@umwilm would you add this to your bug tracker please?


#6

OK, confirm, will be investigated and fixed for next release.


#7

Thanks!
In case you need more test fodder; these are all affected by this issue:

S2A_OPER_PRD_MSIL1C_PDMC_20160417T212609_R050_V20150825T091004_20150825T091004 GranuleNr: 15
S2A_OPER_PRD_MSIL1C_PDMC_20160508T164412_R007_V20160508T090432_20160508T090432 GranuleNr: 10
S2A_OPER_PRD_MSIL1C_PDMC_20160523T030302_R136_V20150801T093004_20150801T093004 GranuleNr: 14
S2A_OPER_PRD_MSIL1C_PDMC_20160523T162813_R079_V20160523T095035_20160523T095035 GranuleNr: 4
S2A_OPER_PRD_MSIL1C_PDMC_20160614T222543_R050_V20160610T090901_20160610T090901 GranuleNr: 11
S2A_OPER_PRD_MSIL1C_PDMC_20160618T074113_R007_V20160617T090020_20160617T090020 GranuleNr: 4
S2A_OPER_PRD_MSIL1C_PDMC_20160707T184040_R136_V20150831T093006_20150831T093006 GranuleNr: 5
S2A_OPER_PRD_MSIL1C_PDMC_20160708T134852_R079_V20160602T095034_20160602T095034 GranuleNr: 5
S2A_OPER_PRD_MSIL1C_PDMC_20160711T221415_R079_V20150827T095442_20150827T095442 GranuleNr: 6
S2A_OPER_PRD_MSIL1C_PDMC_20160718T143320_R021_V20160718T083013_20160718T083013 GranuleNr: 8
S2A_OPER_PRD_MSIL1C_PDMC_20160718T144625_R021_V20160718T083013_20160718T083013 GranuleNr: 4
S2A_OPER_PRD_MSIL1C_PDMC_20160720T191619_R093_V20150808T092005_20150808T092005 GranuleNr: 7
S2A_OPER_PRD_MSIL1C_PDMC_20160728T051848_R050_V20151004T091006_20151004T091006 GranuleNr: 8
S2A_OPER_PRD_MSIL1C_PDMC_20160730T032559_R050_V20151004T091006_20151004T091006 GranuleNr: 11
S2A_OPER_PRD_MSIL1C_PDMC_20160822T171726_R079_V20160821T095032_20160821T095031 GranuleNr: 12
S2A_OPER_PRD_MSIL1C_PDMC_20160825T035842_R079_V20160821T095032_20160821T095031 GranuleNr: 12
S2A_OPER_PRD_MSIL1C_PDMC_20160825T075437_R122_V20160824T100032_20160824T100028 GranuleNr: 9
S2A_OPER_PRD_MSIL1C_PDMC_20160926T220814_R007_V20160925T090012_20160925T090516 GranuleNr: 12
S2A_OPER_PRD_MSIL1C_PDMC_20161001T090012_R050_V20150825T091006_20150825T091004 GranuleNr: 1
S2A_OPER_PRD_MSIL1C_PDMC_20161001T090840_R050_V20150825T091006_20150825T091004 GranuleNr: 1
S2A_OPER_PRD_MSIL1C_PDMC_20161001T151015_R093_V20161001T092022_20161001T092024 GranuleNr: 1
S2A_OPER_PRD_MSIL1C_PDMC_20161001T151635_R093_V20161001T092022_20161001T092024 GranuleNr: 1
S2A_OPER_PRD_MSIL1C_PDMC_20161001T152805_R093_V20161001T092022_20161001T092024 GranuleNr: 1
S2A_OPER_PRD_MSIL1C_PDMC_20161002T171824_R107_V20161002T084802_20161002T085358 GranuleNr: 1
S2A_OPER_PRD_MSIL1C_PDMC_20161005T045036_R079_V20150827T095016_20150827T095011 GranuleNr: 1
S2A_OPER_PRD_MSIL1C_PDMC_20161005T140934_R007_V20161005T085822_20161005T085822 GranuleNr: 1
S2A_OPER_PRD_MSIL1C_PDMC_20161006T161156_R021_V20161006T083002_20161006T083005 GranuleNr: 1
S2A_OPER_PRD_MSIL1C_PDMC_20161009T180414_R064_V20161009T083842_20161009T083841 GranuleNr: 1
S2A_OPER_PRD_MSIL1C_PDMC_20161026T161328_R007_V20161025T090032_20161025T090032 GranuleNr: 1
S2A_OPER_PRD_MSIL1C_PDMC_20161026T162209_R007_V20161025T090032_20161025T090032 GranuleNr: 1
S2A_OPER_PRD_MSIL1C_PDMC_20161028T092900_R021_V20161026T083032_20161026T083032 GranuleNr: 1
S2A_OPER_PRD_MSIL1C_PDMC_20161215T061508_R021_V20151022T083022_20151022T083022 GranuleNr: 8
S2A_OPER_PRD_MSIL1C_PDMC_20161230T145158_R007_V20151001T085756_20151001T085756 GranuleNr: 3
S2A_OPER_PRD_MSIL1C_PDMC_20170101T013615_R050_V20150924T090726_20150924T090726 GranuleNr: 6


#8

Hi bleutner,

can you please attach your configuration? (L2A_GIPP.xml)?
I just tested the two following products:
S2A_OPER_PRD_MSIL1C_PDMC_20170101T013615_R050_V20150924T090726_20150924T090726.SAFE
Granule: S2A_OPER_MSI_L1C_TL_EPA__20161231T221649_A001334_T36UUG_N02.04
S2A_OPER_PRD_MSIL1C_PDMC_20161026T162209_R007_V20161025T090032_20161025T090032 GranuleNr: 1
The error does not occur at my test installation (I freshly installed 2.3.0 from the ESA website).

Especially I need to know the three following parameters you are using:
Aerosol_Type: (RURAL, MARITIME, AUTO)
Mid_Latitude: (SUMMER, WINTER, AUTO )
Ozone_Content: (0, f-k, t-y)

second: at wich resolution does it occur? Already at 60m or at 20m ?
third: do you use a DEM, or not?

fourth: which operating system are you using?

Thanks,
umwilmtest_run_1.rtf (15.5 KB)


#9

Hi,

I also did a clean install now and ran it for this scene and it still breaks:
S2A_OPER_PRD_MSIL1C_PDMC_20161026T162209_R007_V20161025T090032_20161025T090032 GranuleNr: 1

System: Ubuntu 16.10 64bit
S2C version: 2.3.0
Resolution: 20m only
Aerosol: RURAL
Mid_Latitude: SUMMER (AUTO still breaks for some scenes. I’m hoping for 2.4.0 there :wink: )
Ozone: 0
Call: L2A_Process --resolution 20 --GIP_L2A /home/myuser/ssd/L2A_GIPP.xml S2A_OPER_PRD_MSIL1C_PDMC_20161026T162209_R007_V20161025T090032_20161025T090032.SAFE
L2A_Gipp: L2A_GIPP.xml (4.5 KB)
Log file: S2A_USER_PRD_MSIL2A_PDMC_20161026T162209_R007_V20161025T090032_20161025T090032.SAFE.log (5.3 KB)

Best regards,
Benjamin


#10

OK, I’ll check again with the product and configuration you attached and let you know.


#11

I could identify and fix the problem - there was a vector calculated, whereas a scalar should be used. This leads to the error observed under certain conditions. Additionally to this issue, the performance on cirrus must be improved, as it looks pretty bad in your samples. The problem will be fixed in version 2.4, foreseen to be released this spring.


#12

Cool. Thanks!

I was indeed under the impression that cirrus correction is rather weak to non-existent in most cases.
On a similar note: I’ve never found any haze correction. Is it enabled at all in sen2cor?


#13

No. Haze correction is not part of the baseline. Maybe implemented in a later release. Btw: the improvement of the cirrus correction is rather complex, so it is very probable that it will not be part of Version 2.4.0, which is intended to be finished soon.


#14

Hi there. Is there any update on the IndexError problem?


#15

The error has been fixed for Sen2Cor 2.4.0, See Release Note, reference: SIIMPC-897