EarthExplorer + Sen2Cor-02.05.05: product metadata file cannot be read

Optical Toolbox sen2cor
#1

Hi folks,

I have seen a number of users reporting issues processing scenes from circa 2015 with Sen2Cor 02.05.05. Apparently, their issues were gone once they moved the .SAFE directories to an upper-level directory (as a workaround to Windows maximum path limits). However, I do see the very same issue processing certain scenes on Linux: sen2cor fails with “Product metadata file cannot be read.”

This is the directory structure of the granule I am trying to process (S2A_MSIL1C_20150908T141046_N0204_R110_T21LTF_20161014T000432.SAFE), which was downloaded from EarthExplorer:

./DATASTRIP/S2A_OPER_MSI_L1C_DS_EPA__20161013T121640_S20150908T141045_N02.04/S2A_OPER_MTD_L1C_DS_EPA__20161013T121640_S20150908T141045.xml
./GRANULE/S2A_OPER_MSI_L1C_TL_EPA__20161013T121640_A001108_T21LTF_N02.04/IMG_DATA/S2A_OPER_MSI_L1C_TL_EPA__20161013T121640_A001108_T21LTF_B01.jp2
[...]
./GRANULE/S2A_OPER_MSI_L1C_TL_EPA__20161013T121640_A001108_T21LTF_N02.04/QI_DATA/S2A_OPER_MSI_L1C_TL_EPA__20161013T121640_A001108_T21LTF_FORMAT_CORRECTNESS_report.xml
./GRANULE/S2A_OPER_MSI_L1C_TL_EPA__20161013T121640_A001108_T21LTF_N02.04/QI_DATA/S2A_OPER_MSK_CLOUDS_EPA__20161013T121640_A001108_T21LTF_B00_MSIL1C.gml
[...]
./GRANULE/S2A_OPER_MSI_L1C_TL_EPA__20161013T121640_A001108_T21LTF_N02.04/QI_DATA/S2A_OPER_PVI_L1C_TL_EPA__20161013T121640_A001108_T21LTF.jp2
./GRANULE/S2A_OPER_MSI_L1C_TL_EPA__20161013T121640_A001108_T21LTF_N02.04/S2A_OPER_MTD_L1C_TL_EPA__20161013T121640_A001108_T21LTF.xml
./INSPIRE.xml
./S2A_OPER_MTD_SAFL1C_PDMC_20161014T000432_R110_V20150908T141046_20150908T141045.xml
./manifest.safe
./rep_info/S2_User_Product_Level-1C_Metadata.xsd

The code seems to be identifying this granule as “compact format”, but it fails to find a metadata file named MTD_DS.xml, which does not exist in the zip file retrieved from EarthExplorer. The zip file does include a file that matches what’s in a “safe standard” archive (the S2A_OPER_MTD…xml file), but then it does not include the AUX_DATA folder.

Is this an issue with the data, or is this potentially a bug in sen2cor? Is there a way to create the missing metadata so that sen2cor can succeed on processing this scene?

Edit: the content of the zip file looks the same as under Google Cloud Storage:
https://console.cloud.google.com/storage/browser/gcp-public-data-sentinel-2/tiles/20/L/RL/S2A_MSIL1C_20150908T141046_N0204_R110_T20LRL_20161014T000432.SAFE/

Thanks for your attention!

#2

Update: the bug is gone once the tricks from this post are applied. To summarize:

  1. rename the directory from S2A_MSIL1C_… to S2A_OPER_PRD_MSIL1C_…
  2. create the missing directories HTML and AUX_DATA, leaving both with empty contents
  3. re-run Sen2Cor

Please note that once the directory has been renamed to include OPER_PRD, then it is not necessary to modify the following piece from L2A_Tables.py, as suggested in that post:

self.datatakeSensingTime = L2A_UP_ID[25:40] to
self.datatakeSensingTime = L2A_UP_ID[20:35]

Hopefully this follow-up can help somebody else in the future.