Orbit file timeout (March 2021)

@marpet

I actually don’t see a gap (or missing files), when I look at:
http://step.esa.int/auxdata/orbits/Sentinel-1/POEORB/S1A/2020/12/
There are files for every day…

According to the error message, snappy is checking the wrong websites, right? No matter, which date I try…

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-21-f5ec346988b3> in <module>()
      4 parameters.put('Apply-Orbit-File', True)
      5 # apply_orbit = snappy.GPF.createProduct('Apply-Orbit-File', parameters, subset)
----> 6 apply_orbit = snappy.GPF.createProduct('Apply-Orbit-File', parameters, s1_read)
      7 print(colored('Orbit updated succesfully', 'green'))
      8 

RuntimeError: org.esa.snap.core.gpf.OperatorException: No valid orbit file found for 26-DEC-2020 05:23:47.499596
Orbit files may be downloaded from https://qc.sentinel1.eo.esa.int/ or http://aux.sentinel1.eo.esa.int/
and placed in C:\Users\florian.beyer\.snap\auxdata\Orbits\Sentinel-1\POEORB\S1B\2020\12

The error message is just wrong. It does not consider which repository are actually used.

Yes, there are orbits for every day but I think the between 15 and 17 is to long. But this is just a guess. Maybe it is correct.

The same product works in the GUI?

yes, same product is working well in the gui…

Maybe you have an issue similar to the one @CorradoMotta had:

According to his reply it is now working with snappy.

Has someone else an idea?

I already checked that. Everything is fine with the code and the location of that SNAP folder.

I am facing the same problem Orbit file not available error . I am working in HPC where GUI is not an option it is either command line or snappy. I think there is a problem with the integration of snappy with latest sentinel toolbox (which is already upgraded to the latest version)

On my case I don’t think it’s related to my snap location but more in a connection problem with the orbits : i’m using snappy on a server and when it reach the “Apply orbit” processing i got this message :

Exception calling QC Rest API:  Connect to qc.sentinel1.eo.esa.int:443 [qc.sentinel1.eo.esa.int/131.176.235.71] failed: Connection timed out (Connection timed out)
WARNING: org.esa.s1tbx.sar.gpf.orbits.ApplyOrbitFileOp: Connect to qc.sentinel1.eo.esa.int:443 [qc.sentinel1.eo.esa.int/131.176.235.71] failed: Connection timed out (Connection timed out)
Exception calling QC Rest API:  Connect to qc.sentinel1.eo.esa.int:443 [qc.sentinel1.eo.esa.int/131.176.235.71] failed: Connection timed out (Connection timed out)
WARNING: org.esa.s1tbx.sar.gpf.orbits.ApplyOrbitFileOp: ApplyOrbit ignoring error and continuing: org.apache.http.conn.HttpHostConnectException: Connect to qc.sentinel1.eo.esa.int:443 [qc.sentinel1.eo.esa.int/131.176.235.71] failed: Connection timed out (Connection timed out)

Then it ignores it as set up in the parameters but when it reachs the “Terrain correction” processing it retrieve a Null object :

RuntimeError: org.esa.snap.core.gpf.OperatorException: org.esa.snap.core.gpf.OperatorException due to java.lang.NullPointerException

So I decided to try on the GUI on my laptop with latest version as on the server and when i’m trying to apply an orbit file it just doesn’t work so in my opinion it’s all related to the orbit files.

I know that it’s already things that were said in this thread so I don’t think that my comment is really helping…
But if any one has an option to help that would be great as my work is based on S1 products i’m totally stuck right now …

1 Like

how you did it @dmb891 ?

@MaxH
According to your Exception error, your snappy installation tries to get the orbits from the old adress:

I guess, that’s the problem.
And I also think, that is my problem, too.
On SNAP GUI (Toolbox version 8.0.3) the orbit download works after updating.
It seems however that the snappy package was not updated… No idea whether this is the problem or not…

Hi @panggea, I just uninstalled the old SNAP version from the system, subsequently installed the version 8.0 and finally updated it via GUI to the version 8.0.3. After this upgrade SNAP started successfully to download RESORB files from the POD hub.

The problem still exists after updating to V8.0 in Linux.

So, will this be solved with V8.0.3 in Linux and how can I update from V8.0 to V8.0.3?

I am not the Admin of the my system, just want to make sure if the right version is available now and save their time.

Thanks

Have you solved the issue?

I am also a Linux user, now still suffer the orbit time out problem… My version V8.0

How to update to the latest version?

snap --nosplash --nogui --modules --update-all
snap is located in bin folder of snap

I’m still with problems on Linux (Ubuntu 20.04). I have the SNAP version 8.0.3 and I’m trying to run the apply orbit file function on gpt but I’m getting this message:

INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Incompatible GDAL 3.2.1 found on system. Internal GDAL 3.0.0 from distribution will be used.
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Internal GDAL 3.0.0 set to be used by SNAP.
INFO: org.esa.snap.core.util.EngineVersionCheckActivator: Please check regularly for new updates for the best SNAP experience.
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Internal GDAL 3.0.0 set to be used by SNAP.
Executing processing graph
INFO: org.hsqldb.persist.Logger: dataFileCache open start
Exception calling QC Rest API:  Connect to qc.sentinel1.eo.esa.int:443 [qc.sentinel1.eo.esa.int/131.176.235.71] failed: Connection timed out (Connection timed out)
WARNING: org.esa.s1tbx.sar.gpf.orbits.ApplyOrbitFileOp: Connect to qc.sentinel1.eo.esa.int:443 [qc.sentinel1.eo.esa.int/131.176.235.71] failed: Connection timed out (Connection timed out)
Exception calling QC Rest API:  Connect to qc.sentinel1.eo.esa.int:443 [qc.sentinel1.eo.esa.int/131.176.235.71] failed: Connection timed out (Connection timed out)
Connect to qc.sentinel1.eo.esa.int:443 [qc.sentinel1.eo.esa.int/131.176.235.71] failed: Connection timed out (Connection timed out) due to Connection timed out (Connection timed out)
 done.

Error: [NodeId: Apply-Orbit-File] Connect to qc.sentinel1.eo.esa.int:443 [qc.sentinel1.eo.esa.int/131.176.235.71] failed: Connection timed out (Connection timed out) due to Connection timed out (Connection timed out)

How can I solve this issue?

1 Like

Hi @florian.beyer,
have you solved this or do you have some hint for me? I think we are facing the same issue: “Apply orbit file” works on SNAP GUI but it gives problem with snappy.
Thank you in advance.

Salvatore S.

1 Like

Hi @s.savastano,

I’ve found a work around, but I did not have a solution for the actual problem.
My workaround, however, doesn’t work for you except you are working for a federal german organisation :joy: :joy: :joy:
In my case I’m using already preprocessed S1 data on CodeDE (https://code-de.org/de/) within a premium plus account.

My second workaround is now, that I wait for an update, which is solving this issue for snappy.
Thats it… Sorry.

1 Like

Hi, I have the same issue here. It works for the lasted version using the SNAP GUI but not for the gpt process using command line. Something that could we do or just wait for future update?

Hello

We can download manually the orbit data from https://scihub.copernicus.eu/gnss/, however we are experiencing problems with sentinel 1 orbits files automatic download.

We are processing in near real time, so we ask por AUX_RESORB files (‘Sentinel Restituted (Auto Download)’) when calling Apply-Orbit-File gpt operator. The automatic download is unsuccessful, please see one example of the output of the operator is:

b"INFO: org.esa.snap.python.gpf.PyOperatorSpi: Python operator ‘py_sambuca_snap_op’ registered (Python module: ‘sambuca_snap_op’, class: ‘sambuca_snap_op’, root: ‘/home/seasnet/.snap/system/modules/org-esa-sen2coral-sen2coral-inversion.jar’)\nINFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters\nINFO: org.esa.s2tbx.dataio.gdal.GDALVersion: GDAL 3.0.4 found on system. JNI driver will be used.\nINFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Installed GDAL 3.0.4 set to be used by SNAP.\nINFO: org.esa.snap.core.util.EngineVersionCheckActivator: Please check regularly for new updates for the best SNAP experience.\nINFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Installed GDAL 3.0.4 set to be used by SNAP.\nINFO: org.hsqldb.persist.Logger: dataFileCache open start\nWARNING: org.esa.s1tbx.sar.gpf.orbits.ApplyOrbitFileOp: No valid orbit file found for 30-APR-2021 19:04:05.000000\nOrbit files may be downloaded from https://qc.sentinel1.eo.esa.int/ or http://aux.sentinel1.eo.esa.int/\nand placed in /home/seasnet/.snap/auxdata/Orbits/Sentinel-1/RESORB/S1A/2021/04\n\nError: [NodeId: someNodeId] No valid orbit file found for 30-APR-2021 19:04:05.000000\nOrbit files may be downloaded from https://qc.sentinel1.eo.esa.int/ or http://aux.sentinel1.eo.esa.int/\nand placed in /home/seasnet/.snap/auxdata/Orbits/Sentinel-1/RESORB/S1A/2021/04\n"
b’Executing processing graph\nOpenSearch: https://scihub.copernicus.eu/gnss/search?q=platformname:Sentinel-1 AND platformnumber:A AND producttype:AUX_POEORB AND beginposition:[2021-04-01T00:00:000Z TO 2021-04-31T24:00:000Z]\nOpenSearch: 11 total results on 1 pages.\nOpenSearch: https://scihub.copernicus.eu/gnss/search?q=platformname:Sentinel-1 AND platformnumber:A AND producttype:AUX_POEORB AND beginposition:[2021-04-01T00:00:000Z TO 2021-04-31T24:00:000Z]\nOpenSearch: https://scihub.copernicus.eu/gnss/search?q=platformname:Sentinel-1 AND platformnumber:A AND producttype:AUX_POEORB AND beginposition:[2021-05-01T00:00:000Z TO 2021-05-31T24:00:000Z]\nOpenSearch: 0 total results on 1 pages.\nNo valid orbit file found for 30-APR-2021 19:04:05.000000\nOrbit files may be downloaded from https://qc.sentinel1.eo.esa.int/ or http://aux.sentinel1.eo.esa.int/\nand placed in /home/seasnet/.snap/auxdata/Orbits/Sentinel-1/RESORB/S1A/2021/04\n done.\n’

Although the Restituted are requested, it seems that it is looking for Precise

We were able of download for few days the data using the sentinelsat python module (we use this module to download Sentinel 1 and 2 data). However last week, the orbit file download started to fail. The sentinelsat query, get the desired set of products sentinel 1 orbit files, but fails when try to download throughout uuid:

ConnectionError: HTTPSConnectionPool(host=‘dhusfeed.dhus.onda-dias.net’, port=443): Max retries exceeded with url: /gnss/odata/v1/Products(‘3f838f51-38ed-42b3-bf28-adb4e2a6badb’)/$value (Caused by NewConnectionError(’<urllib3.connection.VerifiedHTTPSConnection object at 0x7f16254ca2e0>: Failed to establish a new connection: [Errno -2] Name or service not known’))

In the information get by the query process. In the link appears https://dhusfeed.dhus.onda-dias.net/gnss instead of https://scihub.copernicus.eu/gnss/. Could be this the problem ? We are not able to access the first web adress. The link that shows this query do not correspond to the Download URL used in https://scihub.copernicus.eu/gnss/. Please see an example of the information retrieved by sentinelsat query:

{‘title’: ‘S1A_OPER_AUX_RESORB_OPOD_20210501T060101_V20210501T013304_20210501T045034’,
‘link’: “https://dhusfeed.dhus.onda-dias.net/gnss/odata/v1/Products(‘3f838f51-38ed-42b3-bf28-adb4e2a6badb’)/$value”,
‘link_alternative’: “https://dhusfeed.dhus.onda-dias.net/gnss/odata/v1/Products(‘3f838f51-38ed-42b3-bf28-adb4e2a6badb’)/”,
‘link_icon’: “https://dhusfeed.dhus.onda-dias.net/gnss/odata/v1/Products(‘3f838f51-38ed-42b3-bf28-adb4e2a6badb’)/Products(‘Quicklook’)/$value”,
‘summary’: ‘Date: 2021-05-01T01:33:04Z, Instrument: , Satellite: Sentinel-1, Size: 546.84 KB’,
‘ondemand’: ‘false’,
‘generationdate’: datetime.datetime(2021, 5, 1, 6, 1, 1),
‘beginposition’: datetime.datetime(2021, 5, 1, 1, 33, 4),
‘endposition’: datetime.datetime(2021, 5, 1, 4, 50, 34),
‘ingestiondate’: datetime.datetime(2021, 5, 1, 6, 10, 10, 182000),
‘format’: ‘EOF’,
‘size’: ‘546.84 KB’,
‘platformname’: ‘Sentinel-1’,
‘platformshortname’: ‘S1’,
‘platformnumber’: ‘A’,
‘platformserialidentifier’: ‘1A’,
‘filename’: ‘S1A_OPER_AUX_RESORB_OPOD_20210501T060101_V20210501T013304_20210501T045034.EOF’,
‘producttype’: ‘AUX_RESORB’,
‘filedescription’: ‘NRT POD Restituted Orbit File’,
‘fileclass’: ‘OPER’,
‘creator’: ‘OPOD’,
‘creatorversion’: ‘1.10.3’,
‘identifier’: ‘S1A_OPER_AUX_RESORB_OPOD_20210501T060101_V20210501T013304_20210501T045034’,
‘uuid’: ‘3f838f51-38ed-42b3-bf28-adb4e2a6badb’}

The SNAP sentinel 1 toolbox we are using is up to date (8.0.3).

Regards
Josep

1 Like

Hey jcoca,

I experienced the same issue. The information returned from the query appears to be incorrect, resulting in the “Max retries exceeded” you mentioned when trying to access the provided link. This is the case for both the main and alternative link. As a workaround I ended up just replacing the domain with “https://scihub.copernicus.eu”, it’s a bit hacky, but it made it work (for now :wink:).

Regards,
Rutger

1 Like

Hi

I have decided to use your workaround. It is a little bit weird what is hapenning with the acces to orbit files.

Thank you very much
Josep