Issues with Sen2Cor v2.9 user guide DEM reference

[mcgregorian1]

My labmate and I ran into the following issues when running Sen2Cor v2.9. We found the solution (below) and emailed the issue to ESA (also below) with suggestions to update the user guide, but they redirected us to this forum.

I’m copying the email we sent here in case anyone else either knows why this occurred, or they end up having the same problem.

---

We were running Sen2Cor v2.9 as a standalone installment and we noticed some odd behavior, especially with regard to the DEM. We tried using the Sen2Cor user guide (Date: 2020-11-30), but we found some irregularities.

Below are our recommendations for updating the user guide, followed by examples of the behavior we were seeing.

Recommendations for arguments to basic L2A processing

1. Please make it clear that the DEM folder string path in the L2A_GIPP.xml file (DEM_Directory, DEM_Reference) must have a trailing slash.

2. It appears DEM_Reference is doing the work of DEM_Directory. Can this please be clarified in the User Guide?

Default .xml file behavior
We wanted to note the L2A_GIPP.xml file was not read by the software until we explicitly referenced it using the --L2A argument. The impression from the user guides is that this should be read automatically, but we found that when we updated the file, it did not affect the output until we added in this argument.

Referencing a DEM within the L2A_GIPP.xml file.
After multiple trials, we have determined odd behavior of <DEM_Directory>. We have documented the different scenarios below.

For the sake of these scenarios,

• absolutePathString = “absolute/path/to/folder”
• relativePathString = “path/to/folder”
• DEM is the downloaded SRTM 90m DEM located within the Sen2Cor folder itself.
  • Note that the user guide is not clear on this placement (directory above the Sen2Cor folder? within the Sen2Cor folder? This was not helpful).

Scenario 1

• <DEM_Directory> set to relativePathString
• <DEM_Reference> set to NONE
• <Force_Exit_On_DEM_Error> set to TRUE

Result: Software attempted to download from NONE and failed.

Scenario 2

• <DEM_Directory> set to absolutePathString
• <DEM_Reference> set to NONE
• <Force_Exit_On_DEM_Error> set to TRUE

Result: Software attempted to download from NONE and failed.

Scenario 3

• <DEM_Directory> set to absolutePathString
• <DEM_Reference> also set to absolutePathString
• <Force_Exit_On_DEM_Error> set to TRUE

Result: Software attempted to download from NONE and failed.

Trying to retrieve DEM from URL absolutePathString this may take some time …
[Errno 2] No such file or directory: ‘absolutePathStringsrtm_55_08.zip’

^note how it tried getting the DEM without adding a trailing / (slash) to the pathString

Scenario 4

• <DEM_Directory> set to absolutePathString with trailing slash
• <DEM_Reference> also set to absolutePathString with trailing slash
• <Force_Exit_On_DEM_Error> set to TRUE

Result: Software accessed the DEM and continued as normal

[chiara]

Dears,

I am trying to do the atmospheric and topographical correction with a self-produced DEM, nominated and with the required specifications, similar to the srtm that covers the study area.
The DEM is located in the Sen2Cor Home folder (“C:\Tools\Sen2Cor-02.09.00-win64\dem”), path indicated in the L2A_GIPP.xml, as in “Scenario 4”, but there is no way to make it work:

Could anyone provide me with support to solve this problem?

Many thanks!

[mcgregorian1]

It’s probably not liking you putting the "C: " in front of the file path. Try removing it so you just have “/Tools/Sen2Cor…”, maybe that will work?

[chiara]

With the suggested modifications:
<DEM_Directory>/Tools/Sen2Cor-02.09.00-win64/dem</DEM_Directory>
<DEM_Reference>/Tools/Sen2Cor-02.09.00-win64/dem/</DEM_Reference>

it happens that it blocks when trying to retrieve the DEM:

It seems that the DEM_Directory is always skipped in favor of the DEM_Reference (which, as already noticed, needs the final “/” to work)

[marpet]

I think you have to add a trailing slash to the DEM_Reference path, as in scenario 4.

Or if a URL if required as “unknown url type” indicates, then try:

file:/C:/Tools/Sen2Cor-02.09.00-win64/dem/

That’s a local file URL

[chiara]

Dear @marpet,

even adding the change you indicated sen2cor continues to block as in the image of the previous post, without giving any error,

what could have gone wrong?

Thanks

[marpet]

Sorry, I have no Idea. I have never used sen2cor either.
My suggestion was just a guess, based on your previous posts.

[mcgregorian1]

I’m assuming in your dem folder you have the zipped folders of the SRTM tiles correct? And you’re sure that both the DEM Reference and the Directory have the trailing slash? Your most recent picture does not show a trailing slash on DEM Directory.

[chiara]

Dear @mcgregorian1,

Yes, the dem folder contains the zip file of my dem, named after the STRM tile of the S2 scene, and also the unzipped itself:

I changed the paths as follows:

of the two L2A_GIPP.xml files present at the following paths:

Once launched, the process stops at the following step:

[gnwiii]

It would be helpful if you post actual text rather than screen. You screen images are nearly unreadable on a laptop screen. Posting text makes it possible for others to search the posts.

Your last error “WindowsError: [Error 32] Impossible accedre al file. Il file utilizzato da un altro processo: …” (WindowsError: [Error 32] Unable to access the file. The file used by another process:) in English.

There are Windows tools such as procmon that will tell you what process is using the file. You can try rebooting to make sure no old process is running and try the processing again. I have seen this erroro with large files and AV software that is taking too long to scan the file. Is your “D:” drive a (slow) external disk?

[chiara]

I think if you click on the image it will zoom in for a better view

Yes, I have already tried to restart, but the result is always the same:

C:\Tools\Sen2Cor-02.09.00-win64>Call L2A_Process.bat D:\Users\Chiara\Gran_Paradiso\Sentinel2\T32TLR\Level1\2021\temp_L1C\S2A_MSIL1C_20210605T102021_N0300_R065_T32TLR_20210605T141903.SAFE
Sen2Cor 2.9.0, created: 2020.11.30, supporting Level-1C product version 14.2 - 14.6
Operation mode: TOOLBOX
Processing baseline: 99.99
Application started …
[…]
[Error 32] Impossibile accedere al file. Il file Þ utilizzato da un altro processo: ‘D:\Users\Chiara\Gran_Paradiso\Sentinel2\T32TLR\Level1\2021\temp_L1C\S2A_MSIL2A_20210605T102021_N9999_R065_T32TLR_20210629T125626.SAFE\GRANULE\L2A_T32TLR_A031093_20210605T102737\tmppumaj_\srtm_L2A_T32TLR_A031093_20210605T102737_src.tif’
Traceback (most recent call last):
File “C:\Tools\Sen2Cor-02.09.00-win64\Lib\site-packages\sen2cor\L2A_Process.py”, line 502, in main
if tile.process() == True:
File “C:\Tools\Sen2Cor-02.09.00-win64\Lib\site-packages\sen2cor\L2A_ProcessTileToolbox.py”, line 118, in process
if not self.process_resolution(20):
File “C:\Tools\Sen2Cor-02.09.00-win64\Lib\site-packages\sen2cor\L2A_ProcessTileToolbox.py”, line 148, in process_resolution
return self._process()
File “C:\Tools\Sen2Cor-02.09.00-win64\Lib\site-packages\sen2cor\L2A_ProcessTileToolbox.py”, line 162, in process
if self.preprocess() == False:
File “C:\Tools\Sen2Cor-02.09.00-win64\Lib\site-packages\sen2cor\L2A_ProcessTileToolbox.py”, line 251, in preprocess
if(self.tables.importBandList() == False):
File “C:\Tools\Sen2Cor-02.09.00-win64\Lib\site-packages\sen2cor\L2A_Tables.py”, line 1030, in importBandList
demfile = self.gdalDEM_srtm()
File “C:\Tools\Sen2Cor-02.09.00-win64\Lib\site-packages\sen2cor\L2A_Tables.py”, line 2014, in gdalDEM_srtm
os.remove(srtmf_src)
WindowsError: [Error 32] Cannot access the file. file Þ used by another process: 'D:\Users\Chiara\Gran_Paradiso\Sentinel2\T32TLR\Level1\2021\temp_L1C\S2A_MSIL2A_20210605T102021_N9999_R065_T32TLR_20210629T125626.SAFE\GRANULE\L2A_T32TLR_A031093_20210605T102737\tmppumaj\srtm_L2A_T32TLR_A031093_20210605T102737_src.tif’
Progress[%]: 100.00 : Application terminated with at least one error.

[gnwiii]

Even in text mode the message seems garbled: "file " instead of “file is”, "file " or some similar text. This may a hint that some configuration file has a garbage character.

Can you try using your C: drive for this processing?

Goggle for `Python2 Windows Error 32 "process explorer’ found several cases. One of them might look familar, for example: useful discussion.

[vicencpala]

One possible temporary solution was found and repported in Processing Error due to SRTM Data Conflict - #6 by vicencpala.