SRTM ZIP-files are corrupted or not found

Hi everybody!

I am using the Graph Processing Tool (GPT) of SNAP on Linux to preprocess and subset Sentinel-1 images. This worked well until yesterday midnight. Suddenly there is a problem with new downloaded DEM SRTM auxdata. If the files exist from a processing job in the past for the given subset, everything works fine. If new files are required, SNAP downloads them automatically, but the downloaded ZIP-files are suddenly corrupted and GPT stays in an endless loop trying to unpack the new ZIP-file:

INFO: org.esa.snap.core.dataop.dem.ElevationFile: http retrieving http://step.esa.int/auxdata/dem/SRTMGL1/N40W080.SRTMGL1.hgt.zip
SEVERE: org.esa.snap.core.dataop.dem.ElevationFile: error in opening zip file
INFO: org.esa.snap.core.dataop.dem.ElevationFile: http retrieving http://step.esa.int/auxdata/dem/SRTMGL1/N40W080.SRTMGL1.hgt.zip
SEVERE: org.esa.snap.core.dataop.dem.ElevationFile: error in opening zip file
INFO: org.esa.snap.core.dataop.dem.ElevationFile: http retrieving http://step.esa.int/auxdata/dem/SRTMGL1/N40W080.SRTMGL1.hgt.zip
SEVERE: org.esa.snap.core.dataop.dem.ElevationFile: error in opening zip file
[...]

After killing the GPT process and testing the ZIP-file, I can see that it is corrupted:

~/.snap/auxdata/dem/SRTM 1Sec HGT$ unzip -t N40W080.SRTMGL1.hgt.zip
Archive:  N40W080.SRTMGL1.hgt.zip
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.
unzip:  cannot find zipfile directory in one of N40W080.SRTMGL1.hgt.zip or
        N40W080.SRTMGL1.hgt.zip.zip, and cannot find N40W080.SRTMGL1.hgt.zip.ZIP, period.

If I remove the corrupted ZIP-file and download it manually using wget, the ZIP-file is totally fine:

~/.snap/auxdata/dem/SRTM 1Sec HGT$ rm N40W080.SRTMGL1.hgt.zip
~/.snap/auxdata/dem/SRTM 1Sec HGT$ wget http://step.esa.int/auxdata/dem/SRTMGL1/N40W080.SRTMGL1.hgt.zip
--2020-12-05 13:05:27--  http://step.esa.int/auxdata/dem/SRTMGL1/N40W080.SRTMGL1.hgt.zip
Resolving step.esa.int (step.esa.int)... 37.252.127.80
Connecting to step.esa.int (step.esa.int)|37.252.127.80|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://step.esa.int/auxdata/dem/SRTMGL1/N40W080.SRTMGL1.hgt.zip [following]
--2020-12-05 13:05:27--  https://step.esa.int/auxdata/dem/SRTMGL1/N40W080.SRTMGL1.hgt.zip
Connecting to step.esa.int (step.esa.int)|37.252.127.80|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 9572548 (9.1M) [application/zip]
Saving to: ā€˜N40W080.SRTMGL1.hgt.zipā€™

N40W080.SRTMGL1.hgt.zip                 100%[=============================================================================>]   9.13M  6.50MB/s    in 1.4s

2020-12-05 13:05:28 (6.50 MB/s) - ā€˜N40W080.SRTMGL1.hgt.zipā€™ saved [9572548/9572548]

~/.snap/auxdata/dem/SRTM 1Sec HGT$ unzip -t N40W080.SRTMGL1.hgt.zip
Archive:  N40W080.SRTMGL1.hgt.zip
    testing: N40W080.hgt              OK
No errors detected in compressed data of N40W080.SRTMGL1.hgt.zip.

If I start the GPT process again, no error occurs and the preprocessing and creation of the subset works fine.

I tried this on two machines:
Ubuntu 18.04 with SNAP 7
Ubuntu 20.04 with SNAP 8

The problem occurs on both systems since last night.

Does someone encounter a similar problem or knows what the problem could be?

Thanks!

6 Likes

Yes, same problem here!

Hi, I am trying to execute the Range Doppler Terrain Correction on Sentinel-1 data using SRTM 3Sec DEM data. But, the download process of the DEM data is repeated continuously and the error appears such as the following picture. I just encounter this issue recently, I have no issue like this before. The message log is attached as well.

[messages.log|attachment]

The message log:
(upload://53U0MujG7ZUkQrkq2w9MWBupHpk.log) (76.0 KB)

have you tried SRTM 1Sec instead?

@ABraun Yes, I did. It also raised an error, but with a different circumstance. It raised ā€œerror in opening zip fileā€. When I checked to auxdata directory, the downloaded DEM data just had 1 kB filesize.

Is it possible that your disk is full?

I donā€™t think so.

I have tried to reinstall the software, but the error still remains.

FYI, this is my specification:
Manufacturer: Dell Inc.
Model: Inspiron 5575
Processor: AMD Ryzen 5 2500U with Radeon Vega Mobile Gfx 2.00 GHz
Installed RAM: 8,00 GB (7,64 GB usable)
System type: 64-bit operating system, x64-based processor
OS: Windows 10 Home (10.0.19042 Build 19042)

maybe your firewall blocks the correct download.
Please make sure that port 443 is open in your firewall settings. How to Open a Port on Windows Firewall
Windows and Mac: How to open ports in your firewall

Still raised the error

would it work to manually download the DEM and use it as an external DEM for the terrain correction?

Correct me if I put it in the wrong way.

Got the same problem with downloading of SRTM files in SNAP.

This topic is related to


and

There has been a server restart on Saturday evening. Since then there are problems with fetching SRTM files in SNAP. Manual downloads outside SNAP are working. Seems for me that this is a server issue.

2 Likes

please exctract the data before you use it as an external DEM. SNAP needs it as GeoTiff in WGS84

Hello there! I have the same problem on back-geocoding. I tried to use an external DEM as GeoTiff and I still get an error message.

same problemā€¦???

I think snapā€™s ability to open zip files is broken since yesterday (or so). We get the same error on the ā€œww15mgh_b.zipā€ file (the data for the EGM96 gravitational model):

2020-12-05T19:15:15.651809346Z INFO: org.esa.snap.engine_utilities.download.DownloadableContentImpl: http retrieving http://step.esa.int/auxdata/dem/egm96/ww15mgh_b.zip
2020-12-05T19:15:15.651816240Z error in opening zip file                            
2020-12-05T19:15:15.651823153Z 90% done.                                                                                               
2020-12-05T19:15:15.651829433Z org.esa.snap.core.gpf.OperatorException: error in opening zip file

Downloading and unzipping that same file locally works perfectly fine, thoughā€¦

@lveci could you please have a look?

Dear s1tbx team,

I get this error when computing interferogram using SRTM 1Sec HGT:
SEVERE: org.esa.snap.core.dataop.dem.ElevationFile: error in opening zip file
INFO: org.esa.snap.core.dataop.dem.ElevationFile: http retrieving http://step.esa.int/auxdata/dem/SRTMGL1/N37W010.SRTMGL1.hgt.zip

I tried to access the link http://step.esa.int/auxdata/dem/SRTMGL1/, and find the file N37W010.SRTMGL1.hgt.zip, but I didnā€™t find.

Thank you in advance for your attention

Iā€™m processing interferograms with SLC products from sentinel 1 A, referring ā‰ˆ 1 year (7/26/2019 - 6/26/2020).
Iā€™m using SRTM 1Sec HGT in Back-Geocoding and TopoPhaseRemoval.

7 posts were split to a new topic: SNAP searches non-existent SRTM tiles