Access to orbit files from 1 November 2023

mfitrzyk · November 7, 2023, 11:07am

At the end of October 2023 the Copernicus Access Hub has terminated its operations. The new data access point for Sentinel-1 data and orbit files is now Copernicus Data Space Ecosystem .

Please note that up to now the automatic download of orbit files through Apply-orbit operator was downloading orbit files from the following sources:

The old POD service scihub.copernicus.eu/gnss/odata

and if the files were not found there from

Step.esa.int server where the local copy of all orbit files was created

Starting from 1 November 2023 the Apply-orbit operator, due to the lack of scihub.copernicus.eu files, will download orbit files from STEP server which is fully synchronised with the Copernicus Data Space Ecosystem.

Therefore the users of SNAP 9 will not be influenced by the change of orbit data access point.

mfitrzyk · November 7, 2023, 11:08am

hriston_bg · November 8, 2023, 10:21am

Should we update SNAP?

mfitrzyk · November 8, 2023, 10:24am

You shall always get the latest updates of SNAP since they may include some bug fixes/important changes to SNAP functionality.

hriston_bg · November 8, 2023, 10:29am

@mfitrzyk OK. Thank you for advising.

Wellington · November 19, 2023, 4:16am

Hello. Is it the reason I am getting the error below? And how can I fix it?

Before flood: Layer error: ImageCollection.load: ImageCollection asset ‘COPERNICUS/S1 GRD’ not found (does not exist or caller does not have access).

After flood: Layer error: ImageCollection.load: ImageCollection asset ‘COPERNICUS/S1 GRD’ not found (does not exist or caller does not have access).

tim · November 20, 2023, 9:05pm

@mfitrzyk

SNAP appears to started having trouble querying/download orbit files on November 20.

Might it be related to URIs like these: https://scihub.copernicus.eu/gnss/search?q=platformname:Sentinel-1 AND platformnumber:A AND producttype:AUX_POEORB AND beginposition:[2023-11-01T00:00:000Z TO 2023-11-31T24:00:000Z] not being a valid query URL anymore?

Jialin · November 21, 2023, 3:55am

@mfitrzyk
But In my case, the snap seems did not fallback to step.eas.int after failure to fetch scihub.copernicus.eu/gnss/odata.
I did following tests:

Using AppyOrbitFile and option of Sentinel Precise(Auto Download) for two S1A zip files (both gui menu and gpt command give same result ) - one from old 2021 and one from Oct 2, 2023. For 2021, it just failed and complained about the failure to fetch orbit file from old url. For the latter one, it just default to download the RESORB files.(I checked the ~/.snap/auxdata/Orbits directory manually)

How exactly the fallback url worked? Did you just do some url auto-redirect or make some modification in the recent update of s1box plugin?

Software version details:
The software and plugins are up-to-date (Nov. 21, 2023). The About SNAP… gives 9.0.0 for SNAP and 9.03 for S1TBX. Further details for SNAP:

SNAP Desktap implementation version: 9.0.8
SNAP Engine implementation version: 9.0.8
JRE: OpenJDK Runtime Environment 1.8.0_242-b20
JVM: OpenJDK 64-Bit ServerVM by Azul Systems,Inc
Memory:79995 MiB

Gerald · November 21, 2023, 7:47am

Same problem here!

diana_harosa · November 21, 2023, 9:02am

Hi @Jialin,
There is not update to be installed for the s1tbx plugin. If the orbit file cannot be downloaded from the scihub.copernicus.eu, SNAP searches for it in our repository which should be fully synchronized with the Copernicus Data Space Ecosystem.
We are currently investigating why the orbits are not downloaded. We’ll keep you updated on any developments.

vchaparro · November 21, 2023, 4:21pm

Hi,
I have the same issue, and I also have a question: will the fix also address the problem for other cases where it tries to access URLs that are no longer available, not just for the apply orbit file operation?

Thank you in advance!

JulienPiquot · November 21, 2023, 11:12pm

Hello Diana,

It seems the text part of the hyperlinks are being mangled on step website.

According to the OrbitFileScraper class, the text part of the hyperlink should end with .zip which doesn’t seem to be the case for some reason:

Regards

geofisue · November 22, 2023, 1:29pm

Do we need to change anything in SNAP? I use SNAP9 up to date, but there is an error when downloading orbit files…

diana_harosa · November 22, 2023, 1:47pm

We are currently experiencing some issues with the orbit files download. Our team is working on a fix. We apologize for the inconvenience and appreciate your patience.
As a workaround, the orbit files can be downloaded using the SNAP Product Library :

and copied to the Orbits folder:

C:\Users<user>\.snap\auxdata\Orbits\Sentinel-1\POEORB\S1A\<year>\<month>

diana_harosa · November 22, 2023, 2:32pm

Dear users,
Sentinel-1 Toolbox version 9.0.4 is now available. Please check for updates using the Help → Check for Updates menu.

andre.theron · November 22, 2023, 2:54pm

We are manually downloading Orbit files and need to switch to the new Copernicus data service but we are unable to authenticate the orbit download request with the token we get. This is an issue since we cannot update our application to SNAP 9 yet. Will the orbit update be backported to SNAP 8?

I have asked on the Copernicus spacedata forums - but the issue might be relevant here too: “Unauthorized OAuth token (401) when downloading S1 orbits from catalogue.dataspace.copernicus.eu/download”

vchaparro · November 22, 2023, 4:10pm

Hi @diana_harosa ,

I have updated to 9.0.4 and still the same problem occurs, it doesn’t find the orbit files.

diana_harosa · November 22, 2023, 4:13pm

Hi @vchaparro,
Did you restart SNAP after this module update?

diana_harosa · November 22, 2023, 4:23pm

Hello @andre.theron,
There is no update planned for SNAP 8.

vchaparro · November 22, 2023, 6:52pm

Hi @diana_harosa ,
I’m using snap from the command line. It does not try to download the file from Step.esa.int server after trying the old scihub.copernicus.eu/gnss/odata. I did update using the command:

snap --nosplash --nogui --modules --update-all

and checked it was updated using

snap --nosplash --nogui --modules --list --refresh

gpt Apply-Orbit-File -e -PcontinueOnFail=false -PorbitType='Sentinel Precise (Auto Download)' -Ssource=/var/seda-satellite/imagery/sentinel/sentinel-1/S1A_IW_GRDH_1SDV_20210406T190539_20210406T190604_037334_046624_AB03.SAFE -SsourceProduct=/var/seda-satellite/imagery/sentinel/sentinel-1/S1A_IW_GRDH_1SDV_20210406T190539_20210406T190604_037334_046624_AB03.SAFE -t /app/notebooks/tmp/S1A_IW_GRDH_1SDV_20210406T190539_20210406T190604_037334_046624_AB03/S1A_IW_GRDH_1SDV_20210406T190539_20210406T190604_037334_046624_AB03_ORB.dim 
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Incompatible GDAL 3.8.0 found on system. Internal GDAL 3.2.1 from distribution will be used.
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Incompatible GDAL 3.8.0 found on system. Internal GDAL 3.2.1 from distribution will be used.
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Incompatible GDAL 3.4.1 found on system. Internal GDAL 3.2.1 from distribution will be used.
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Incompatible GDAL 3.4.1 found on system. Internal GDAL 3.2.1 from distribution will be used.
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Internal GDAL 3.2.1 set to be used by SNAP.
INFO: org.esa.snap.core.util.EngineVersionCheckActivator: Please check regularly for new updates for the best SNAP experience.
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Internal GDAL 3.2.1 set to be used by SNAP.
SEVERE: org.esa.s1tbx.io.sentinel1.Sentinel1Level1Directory: measurement/s1a-iw-grd-vh-20210406t190539-20210406t190604-037334-046624-002_geocoding.tiff not found
SEVERE: org.esa.s1tbx.io.sentinel1.Sentinel1Level1Directory: measurement/s1a-iw-grd-vh-20210406t190539-20210406t190604-037334-046624-002_geocoding_scale.tiff not found
SEVERE: org.esa.s1tbx.io.sentinel1.Sentinel1Level1Directory: measurement/s1a-iw-grd-vv-20210406t190539-20210406t190604-037334-046624-001_geocoding.tiff not found
SEVERE: org.esa.s1tbx.io.sentinel1.Sentinel1Level1Directory: measurement/s1a-iw-grd-vv-20210406t190539-20210406t190604-037334-046624-001_geocoding_scale.tiff not found
INFO: org.hsqldb.persist.Logger: dataFileCache open start
OpenSearch: https://scihub.copernicus.eu/gnss/search?q=platformname:Sentinel-1 AND platformnumber:A AND producttype:AUX_POEORB AND beginposition:[2021-04-01T00:00:000Z TO 2021-04-31T24:00:000Z]
WARNING: org.esa.s1tbx.orbits.gpf.ApplyOrbitFileOp: No valid orbit file found for 06-APR-2021 19:04:35.000000
Orbit files may be downloaded from https://scihub.copernicus.eu/gnss/odata/v1/
and placed in /root/.snap/auxdata/Orbits/Sentinel-1/POEORB/S1A/2021/04
OpenSearch: https://scihub.copernicus.eu/gnss/search?q=platformname:Sentinel-1 AND platformnumber:A AND producttype:AUX_RESORB AND beginposition:[2021-04-01T00:00:000Z TO 2021-04-31T24:00:000Z]
No valid orbit file found for 06-APR-2021 19:04:35.000000