Issues managing S1 Orbit files

SvH · June 8, 2016, 10:23am

Dear developers,
we are faced with some issues when trying to setup our S1 processing chain in a distributed fashion. In particular, we would like to store the SNAP auxdata on a network share, so that it can be shared between multiple instances of SNAP-GPT running in different machines. For this we have made some changes to /etc/snap.auxdata.properties, uncommenting the line defining the “AuxDataPath” and setting it to our desired network location. Since “AuxDataPath” is used numerous times as a parameters in the remaining lines of the /etc/snap.auxdata.properties file, defining things like

OrbitFiles.sentinel1POEOrbitPath = {AuxDataPath}/Orbits/Sentinel-1/POEORB OrbitFiles.sentinel1RESOrbitPath = {AuxDataPath}/Orbits/Sentinel-1/RESORB

we would expect that this single change would do the trick for us. However, when running the GPT, this setting seems to be ignored and the auxdata is just placed in its default location in the .snap/auxdata folder…

The second thing we noticed is that, the location from which the toolbox automatically downloads the orbit files

Index of /auxdata/orbits/Sentinel-1/POEORB/
Index of /auxdata/orbits/Sentinel-1/RESORB/

groups the orbits per months and consequently does not provide recent orbit files. In fact we saw that, doing an apply orbit on a S1A product from June, the toolbox downloaded all Orbit files from May to the auxdata location and prints the following info and warnings:

WARNING: org.esa.snap.engine_utilities.datamodel.DownloadableContentImpl: /home/sarccd/.snap/auxdata/Orbits/Sentinel-1/RESORB/2016/2016-6.zip (No such file or directory)
INFO: org.esa.snap.engine_utilities.datamodel.DownloadableContentImpl: http retrieving http://step.esa.int/auxdata/orbits/Sentinel-1/RESORB/2016-5.zip
WARNING: org.esa.s1tbx.sar.gpf.orbits.ApplyOrbitFileOp: No valid orbit file found for 02-JUN-2016 17:23:19.053102
Orbit files may be downloaded from https://qc.sentinel1.eo.esa.int/
and placed in /home/sarccd/.snap/auxdata/Orbits/Sentinel-1/RESORB/2016
INFO: org.esa.snap.engine_utilities.datamodel.DownloadableContentImpl: http retrieving http://step.esa.int/auxdata/orbits/Sentinel-1/POEORB/2016-6.zip
SEVERE: org.esa.snap.engine_utilities.datamodel.DownloadableContentImpl: Failed to download remote file.
java.io.FileNotFoundException: http://step.esa.int/auxdata/orbits/Sentinel-1/POEORB/2016-6.zip
…

So we strongly wonder if this is desired behavior? Why download the Orbits from the month before? And why try download from Index of /auxdata/orbits/Sentinel-1/RESORB/ if this resource is not ideal (grouping per month and not up-to-date) while you guys have the better alternative to download directly from https://qc.sentinel1.eo.esa.int/ ?

Best regards,
Sven.

lveci · June 8, 2016, 11:28am

The auxdata path is defined by the system. You would need to put the whole .snap in a shared drive. I believe this can be done in the snap.conf file in snap install/etc
If the precise orbits are not found then it looks for the restitute orbits. The logic to try to find the orbit in the previous month should probably only apply if the date is at the start of the month.
As for scraping the qc website, this is planned to be done for next year. It’s just waiting on funding.

SvH · June 8, 2016, 2:28pm

Ok, so it is save to share the entire .snap between SNAP instances? We did not feel confident to do this since the .snap folder also contains folders like “system”. Especially if we end up running different versions of snap I was afraid this might go wrong… Maybe @marpet can comment on this?

As a temporary solution we have created a symlink /home/$User/.snap/auxdata pointing at our network share auxdata folder, and this seems to work for us at the moment.

Sven.

marpet · June 8, 2016, 2:40pm

Sven, you are right. It is not completely safe to share the .snap folder, Especially across SNAP versions. You can provide the user directory as parameter to SNAP.
When you call it use the parameter
--userdir "C:\temp\snapuser"
For gpt you can also use -Dsnap.userdir="/temp/path" I think. But you probably have to change the value in the snap.properties file in the etc folder of the snap installation.

SvH · June 8, 2016, 7:36pm

Actually I don’t want to change the location of the .snap folder, I just want the auxdata folder inside the .snap folder to be on a network location, while the .snap folder itself, including the other sub folders, remain at the default location in the UNIX home directory. The structure of the snap.auxdata.properties inside /etc seems to indicate that this is possible, but changing the AuxDataPath in there does not seem to have any effect.

Sven.

marpet · June 8, 2016, 7:46pm

OK, now I got.
In general it should be possible to define a property which defines the location of the auxdata. What do you think @lveci?

Piotr · April 26, 2019, 8:39am

Where can I find this property? It’s already implemented?

lidehua · May 16, 2022, 12:25pm

Hello @lveci SNAP shows wrong date when adding precision orbit information

3201 is obviously wrong, I know the cause of this situation, how can I solve this problem, thanks

sumart · June 29, 2023, 3:36am

Hi!
I got the same problem with wrong date when adding precision orbit information.
Do you solve it?
Please let me know! thanks!