Auto download of DEMs duplicated in different threads?

Microwave Toolbox
#1

This appears to be a bug possibly related to multiple threads encountering the need for the same DEM at about the same time @jun_lu?. I originally noticed a duplicate download notice during processing:

Copernicus_DSM_COG_10_N01_00_W074_00_DEM.tif
Copernicus_DSM_COG_10_N01_00_W075_00_DEM.tif
Copernicus_DSM_COG_10_N02_00_W074_00_DEM.tif
Copernicus_DSM_COG_10_N02_00_W075_00_DEM.tif
Copernicus_DSM_COG_10_N01_00_W073_00_DEM.tif
Copernicus_DSM_COG_10_N00_00_W074_00_DEM.tif
**Copernicus_DSM_COG_10_N00_00_W074_00_DEM.tif**
Copernicus_DSM_COG_10_N02_00_W073_00_DEM.tif
Copernicus_DSM_COG_10_N00_00_W073_00_DEM.tif

Switching to SRTM I also noticed duplicate download notices, but worse was the timing of the download must have been such to produce an error as one thread stomped on another’s download?

INFO: org.esa.snap.core.dataop.dem.ElevationFile: http retrieving http://step.esa.int/auxdata/dem/SRTMGL1/N01W074.SRTMGL1.hgt.zip
INFO: org.esa.snap.core.dataop.dem.ElevationFile: http retrieving http://step.esa.int/auxdata/dem/SRTMGL1/N02W074.SRTMGL1.hgt.zip
INFO: org.esa.snap.core.dataop.dem.ElevationFile: http retrieving http://step.esa.int/auxdata/dem/SRTMGL1/N01W075.SRTMGL1.hgt.zip
INFO: org.esa.snap.core.dataop.dem.ElevationFile: http retrieving http://step.esa.int/auxdata/dem/SRTMGL1/N02W075.SRTMGL1.hgt.zip
INFO: org.esa.snap.core.dataop.dem.ElevationFile: http retrieving http://step.esa.int/auxdata/dem/SRTMGL1/N01W073.SRTMGL1.hgt.zip
INFO: org.esa.snap.core.dataop.dem.ElevationFile: http retrieving http://step.esa.int/auxdata/dem/SRTMGL1/N02W073.SRTMGL1.hgt.zip
INFO: org.esa.snap.core.dataop.dem.ElevationFile: http retrieving http://step.esa.int/auxdata/dem/SRTMGL1/N00W074.SRTMGL1.hgt.zip
INFO: org.esa.snap.core.dataop.dem.ElevationFile: http retrieving http://step.esa.int/auxdata/dem/SRTMGL1/N00W075.SRTMGL1.hgt.zip
SEVERE: org.esa.snap.core.dataop.dem.ElevationFile: error in opening zip file
INFO: org.esa.snap.core.dataop.dem.ElevationFile: http retrieving http://step.esa.int/auxdata/dem/SRTMGL1/N00W074.SRTMGL1.hgt.zip
SEVERE: org.esa.snap.core.dataop.dem.ElevationFile: error in opening zip file
INFO: org.esa.snap.core.dataop.dem.ElevationFile: http retrieving http://step.esa.int/auxdata/dem/SRTMGL1/N00W075.SRTMGL1.hgt.zip
INFO: org.esa.snap.core.dataop.dem.ElevationFile: http retrieving http://step.esa.int/auxdata/dem/SRTMGL1/N00W073.SRTMGL1.hgt.zip

My process is noted in this post.

#3

I don’t see duplicated download here. All the DEM tiles are different. The only place where duplicate download message is seen is when the previous download failed and it tries to download the DEM tile again, like in the case of N00W074.SRTMGL1.hgt.zip and N00W075.SRTMGL1.hgt.zip above.

#4

Copernicus_DSM_COG_10_N00_00_W074_00_DEM.tif is duplicated (I tried to make it bold, but it has **) in the Copernicus download.
I don’t see any message of a download failure for the SRTM. I think a second download of the DEM started, just as the first thread attempted to access the file resulting in the “error opening zip file”.
I am just guessing, you will know better. Thank you.