Response code: 403 on SRTM

Microwave Toolbox Problem Reports
#1

Hi,

This morning I’m having this issue using GPT on s1tbx terrain correction:

SEVERE: org.esa.snap.core.dataop.dem.ElevationFile: Server returned HTTP response code: 403 for URL: https://cgiar-csi-srtm.openterrain.org.s3.amazonaws.com/source/srtm_35_11.zip

This was working just fine yesterday and I didn’t change anything to my code.

Is the file missing ?

Regards,
Stephan

#2

probably related to this: SRTM ZIP-files are corrupted or not found

#3

It may be related but now it’s an access denied error, and it was perfectly working yesterday !

Is there some maintenance going on this dataset ?
It seems to be on AWS S3 so it should be reliable…

Gpt sentinel-1 back-geocoding error
#4

Same problem here. I get that error even if I’m using a non-SRTM DEM for SAR simulation

#5

Same issue here. We have an automated pipeline that fetches and runs snap on S1 data. There were no changes on ours side in the last few weeks and 2 days ago all jobs started failing. Here’s the Snap output:

# /home/snapper/snap/bin/gpt -e S1A_IW_GRDH_1SDV_20201208T230720_20201208T230745_035601_0429F7_AFAA.out/S1A__IW___A_20201208T230720_NR_Orb_TC/temp/S1A_IW_GRDH_1SDV_20201208T230720_20201208T230745_035601_0429F7_AFAA_tmp0.xml 
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.esa.snap.core.util.EngineVersionCheckActivator: Please check regularly for new updates for the best SNAP experience.
Executing processing graph
INFO: org.hsqldb.persist.Logger: dataFileCache open start
WARNING: org.esa.s1tbx.io.orbits.sentinel1.StepAuxdataScraper: Unable to connect to http://step.esa.int/auxdata/orbits/Sentinel-1/POEORB/S1A/2020/12/: HTTP error fetching URL
WARNING: org.esa.s1tbx.sar.gpf.orbits.ApplyOrbitFileOp: No valid orbit file found for 08-DEC-2020 23:06:14.981421
Orbit files may be downloaded from https://qc.sentinel1.eo.esa.int/
and placed in /root/.snap/auxdata/Orbits/Sentinel-1/POEORB/S1A/2020/12
WARNING: org.esa.s1tbx.sar.gpf.orbits.ApplyOrbitFileOp: Using Sentinel Restituted /root/.snap/auxdata/Orbits/Sentinel-1/RESORB/S1A/2020/12/S1A_OPER_AUX_RESORB_OPOD_20201209T034334_V20201208T225644_20201209T021414.EOF.zip instead
INFO: org.esa.snap.core.dataop.dem.ElevationFile: http retrieving http://cgiar-csi-srtm.openterrain.org.s3.amazonaws.com/source/srtm_21_04.zip
INFO: org.esa.snap.core.dataop.dem.ElevationFile: http retrieving http://srtm.csi.cgiar.org/wp-content/uploads/files/srtm_5x5/TIFF/srtm_21_04.zip
INFO: org.esa.snap.core.dataop.dem.ElevationFile: http retrieving http://cgiar-csi-srtm.openterrain.org.s3.amazonaws.com/source/srtm_21_04.zip
SEVERE: org.esa.snap.core.dataop.dem.ElevationFile: Server returned HTTP response code: 403 for URL: http://cgiar-csi-srtm.openterrain.org.s3.amazonaws.com/source/srtm_21_04.zip
INFO: org.esa.snap.core.dataop.dem.ElevationFile: http retrieving http://cgiar-csi-srtm.openterrain.org.s3.amazonaws.com/source/srtm_21_04.zip
INFO: org.esa.snap.core.dataop.dem.ElevationFile: http retrieving http://srtm.csi.cgiar.org/wp-content/uploads/files/srtm_5x5/TIFF/srtm_21_04.zip
INFO: org.esa.snap.core.dataop.dem.ElevationFile: http retrieving http://cgiar-csi-srtm.openterrain.org.s3.amazonaws.com/source/srtm_21_04.zip
SEVERE: org.esa.snap.core.dataop.dem.ElevationFile: Server returned HTTP response code: 403 for URL: http://cgiar-csi-srtm.openterrain.org.s3.amazonaws.com/source/srtm_21_04.zip
INFO: org.esa.snap.core.dataop.dem.ElevationFile: http retrieving http://cgiar-csi-srtm.openterrain.org.s3.amazonaws.com/source/srtm_21_04.zip
INFO: org.esa.snap.core.dataop.dem.ElevationFile: http retrieving http://srtm.csi.cgiar.org/wp-content/uploads/files/srtm_5x5/TIFF/srtm_21_04.zip
INFO: org.esa.snap.core.dataop.dem.ElevationFile: http retrieving http://cgiar-csi-srtm.openterrain.org.s3.amazonaws.com/source/srtm_21_04.zip
SEVERE: org.esa.snap.core.dataop.dem.ElevationFile: Server returned HTTP response code: 403 for URL: http://cgiar-csi-srtm.openterrain.org.s3.amazonaws.com/source/srtm_21_04.zip
INFO: org.esa.snap.core.dataop.dem.ElevationFile: http retrieving http://cgiar-csi-srtm.openterrain.org.s3.amazonaws.com/source/srtm_21_04.zip
INFO: org.esa.snap.core.dataop.dem.ElevationFile: http retrieving http://srtm.csi.cgiar.org/wp-content/uploads/files/srtm_5x5/TIFF/srtm_21_04.zip
INFO: org.esa.snap.core.dataop.dem.ElevationFile: http retrieving http://cgiar-csi-srtm.openterrain.org.s3.amazonaws.com/source/srtm_21_04.zip
SEVERE: org.esa.snap.core.dataop.dem.ElevationFile: Server returned HTTP response code: 403 for URL: http://cgiar-csi-srtm.openterrain.org.s3.amazonaws.com/source/srtm_21_04.zip
INFO: org.esa.snap.core.dataop.dem.ElevationFile: http retrieving http://cgiar-csi-srtm.openterrain.org.s3.amazonaws.com/source/srtm_21_04.zip
INFO: org.esa.snap.core.dataop.dem.ElevationFile: http retrieving http://srtm.csi.cgiar.org/wp-content/uploads/files/srtm_5x5/TIFF/srtm_21_04.zip

Worse is that Snap doesn’t exit, it just sits indefinitely until our container times out.

1 Like
#6

This only happens when I’m processing a scene withing SRTM’s bounds. Anything outside of 60N/S will work fine. Running with -Dsnap.gpf.allowAuxdataDownload=false has no effect.

#7

It’s really nothing to do with SNAP, the openterrain data has had the bucket locked down, it’s Access Denied for the entire bucket.

#8

Could it fall back on http://step.esa.int/auxdata/dem/SRTMGL1/? It seems to try and fetch those models whether or not I’m using SRTM, does it with ACE30 and GETASSE30 too.