Unable to Run Apply-Orbit-File from GraphProcessor

Hi,

I have a Graph that I am trying to integrate into a Java application. I tested the graph using SNAP Desktop and it works, I tested it using gpt and it works. I have some working code integrating the execution of a Graph into my application and it is able to run simple graphs using the GraphProcessor class and executeGraph methods.
However, when I include the “Apply-Orbit-File” operator, it always crashes with the following error:

INFO: org.esa.snap.engine_utilities.download.downloadablecontent.DownloadableContentImpl: http retrieving http://step.esa.int/auxdata/orbits/***
SEVERE: org.esa.snap.engine_utilities.download.downloadablecontent.DownloadableContentImpl: Unable to create folders in \\S1B****
SEVERE: org.esa.snap.engine_utilities.download.downloadablecontent.DownloadableContentImpl: Failed to download remote file.

It looks to me that the code is not able to determine the location where to download the Orbit files, although OrbitFiles.sentinel1POEOrbitPath is set in the snap.auxdata.properties.

I tried setting AuxDataPath (commented by default) but I have the same error. I tried navigating a bit the code for s1tbx and it seems that the actual value for the “prefOrbitPath” is read from the settings, so I guess it should be fine, but I am not able to make it work.

So basically I have a couple of questions:

  1. Is it the expected behavior? Am I missing to configure something before running the graph?. The section of the code for running the graph is pretty straight-forward:

GraphProcessor processor = new GraphProcessor();
Graph graph;
try(Reader fileReader = new FileReader(filePath)) {
graph = GraphIO.read(fileReader);
}
Product[] products = processor.executeGraph(new GraphContext(graph), pm);

  1. Is there a way to override the settings through code so I can try specifying the “OrbitFiles.sentinel1POEOrbitPath” in a different way?

Thank you very much. Any help would be really appreciated.

Anyone with the same problem?

I am also working on integrating gpt into an application and I am having a similar problem with both orbit files and DEMs. I have followed the download URLs output by the console and they don’t work. This is on a linux machine. Right now I am trying to mess around with aux-data folder path which doesn’t seem to have actually been generated yet.

WARNING: org.esa.s1tbx.io.orbits.sentinel1.StepAuxdataScraper: Unable to connect to     http://step.esa.int/auxdata/orbits/Sentinel-1/POEORB/S1B/2018/06/: HTTP error fetching URL
WARNING: org.esa.s1tbx.io.orbits.sentinel1.StepAuxdataScraper: Unable to connect to http://step.esa.int/auxdata/orbits/Sentinel-1/POEORB/S1B/2018/07/: HTTP error fetching URL
WARNING: org.esa.s1tbx.sar.gpf.orbits.ApplyOrbitFileOp: No valid orbit file found for 21-JUN-2018 05:33:15.061865
Orbit files may be downloaded from https://qc.sentinel1.eo.esa.int/
and placed in /root/.snap/auxdata/Orbits/Sentinel-1/POEORB/S1B/2018/06
WARNING: org.esa.s1tbx.io.orbits.sentinel1.StepAuxdataScraper: Unable to connect to http://step.esa.int/auxdata/orbits/Sentinel-1/RESORB/S1B/2018/06/: HTTP error fetching URL
WARNING: org.esa.s1tbx.io.orbits.sentinel1.StepAuxdataScraper: Unable to connect to http://step.esa.int/auxdata/orbits/Sentinel-1/RESORB/S1B/2018/07/: HTTP error fetching URL
INFO: org.esa.snap.engine_utilities.datamodel.DownloadableContentImpl: http retrieving https://qc.sentinel1.eo.esa.int/aux_resorb/S1A_OPER_AUX_RESORB_OPOD_20180621T064436_V20180621T023024_20180621T054754
SEVERE: org.esa.snap.engine_utilities.datamodel.DownloadableContentImpl: Failed to download remote file.
java.io.FileNotFoundException: https://qc.sentinel1.eo.esa.int/aux_resorb/S1A_OPER_AUX_RESORB_OPOD_20180621T064436_V20180621T023024_20180621T054754
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1890)
        at sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1885)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1884)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1457)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254)
        at org.esa.snap.engine_utilities.datamodel.DownloadableContentImpl.downloadFile(DownloadableContentImpl.java:247)
        at org.esa.snap.engine_utilities.datamodel.DownloadableContentImpl.lambda$getRemoteHttpFile$0(DownloadableContentImpl.java:224)
        at org.esa.snap.engine_utilities.datamodel.DownloadableContentImpl.getRemoteHttpFile(DownloadableContentImpl.java:229)
        at org.esa.s1tbx.io.orbits.sentinel1.SentinelPODOrbitFile.getQCFiles(SentinelPODOrbitFile.java:314)
        at org.esa.s1tbx.io.orbits.sentinel1.SentinelPODOrbitFile.downloadFromQCWebsite(SentinelPODOrbitFile.java:166)
        at org.esa.s1tbx.io.orbits.sentinel1.SentinelPODOrbitFile.retrieveOrbitFile(SentinelPODOrbitFile.java:112)
        at org.esa.s1tbx.sar.gpf.orbits.ApplyOrbitFileOp.updateOrbits(ApplyOrbitFileOp.java:281)
        at org.esa.s1tbx.sar.gpf.orbits.ApplyOrbitFileOp.initialize(ApplyOrbitFileOp.java:207)
        at org.esa.snap.core.gpf.internal.OperatorContext.initializeOperator(OperatorContext.java:486)
        at org.esa.snap.core.gpf.internal.OperatorContext.getTargetProduct(OperatorContext.java:273)
        at org.esa.snap.core.gpf.Operator.getTargetProduct(Operator.java:387)
        at org.esa.snap.core.gpf.graph.NodeContext.initTargetProduct(NodeContext.java:77)
        at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:195)
        at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:178)
        at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:178)
        at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:178)
        at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:178)
        at org.esa.snap.core.gpf.graph.GraphContext.initOutput(GraphContext.java:162)
        at org.esa.snap.core.gpf.graph.GraphContext.<init>(GraphContext.java:91)
        at org.esa.snap.core.gpf.graph.GraphContext.<init>(GraphContext.java:64)
        at org.esa.snap.core.gpf.graph.GraphProcessor.executeGraph(GraphProcessor.java:130)
        at org.esa.snap.core.gpf.main.DefaultCommandLineContext.executeGraph(DefaultCommandLineContext.java:86)
        at org.esa.snap.core.gpf.main.CommandLineTool.executeGraph(CommandLineTool.java:534)
        at org.esa.snap.core.gpf.main.CommandLineTool.runGraph(CommandLineTool.java:388)
        at org.esa.snap.core.gpf.main.CommandLineTool.runGraphOrOperator(CommandLineTool.java:287)
        at org.esa.snap.core.gpf.main.CommandLineTool.run(CommandLineTool.java:188)
        at org.esa.snap.core.gpf.main.CommandLineTool.run(CommandLineTool.java:121)
        at org.esa.snap.core.gpf.main.GPT.run(GPT.java:54)
        at org.esa.snap.core.gpf.main.GPT.main(GPT.java:34)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.esa.snap.runtime.Launcher.lambda$run$0(Launcher.java:55)
        at org.esa.snap.runtime.Engine.runClientCode(Engine.java:189)
        at org.esa.snap.runtime.Launcher.run(Launcher.java:51)
        at org.esa.snap.runtime.Launcher.main(Launcher.java:31)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.exe4j.runtime.LauncherEngine.launch(LauncherEngine.java:62)
        at com.install4j.runtime.launcher.UnixLauncher.main(UnixLauncher.java:57)

It seems that no new orbit files where uploaded to the step.esa.int server since mid May.
http://step.esa.int/auxdata/orbits/Sentinel-1/POEORB/S1B/2018

Maybe some service is not yet running on the new server?
@ramona_manda @lveci can you check this?

However, the download from Sentinel-1 QC should still work. But I noticed that the requested file only exists on the server with the suffix “.EOF”. That’s why it can’t be found.
https://qc.sentinel1.eo.esa.int/aux_resorb/S1A_OPER_AUX_RESORB_OPOD_20180621T064436_V20180621T023024_20180621T054754
Has this changed in the recent past?
@mengdahl, @lveci do you know something about it?

In addition, I wonder why on the step.esa.int server a S1B file is requested but on the QC server a S1A.
@mengdahl, @lveci do you know something about it?

You are right that for the last months they are not available. This happens both for gpt (command line interface) and through GraphProcessor (Java code).

However, at least in my case, the issue is there even if the Orbit file exists. I can see that the URL to be retrieved works when copy-pasting it to the browser.
The problem is that running the graph through GraphProcessor does not take some settings into account in the same way gpt does. As a result, it tries to download the Orbit files to an invalid location (\S1B\2017\08) that does not exist (missing the prefix of the path).

Ah, now I saw that you have written your own java code.
What is the working directory when you run your code. It should be the snap installation folder. Actually, it depends on how you have setup your application.
I’m not sure, I haven’t tested, but I think with the following code you can provide the settings in your code and override the defaults.

//org.esa.snap.core.util.SystemUtils
SystemUtils.init3rdPartyLibs(null);
// org.esa.snap.runtime.Engine
Engine.start()
// org.esa.snap.engine_utilities.util.Settings
Settings.instance(); // just to init the auxdata settings
// org.esa.snap.runtime.Config
Preferences auxdataPreferences = Config.instance("snap.auxdata").preferences();
auxdataPreferences.put("AuxDataPath", SystemUtils.getAuxDataPath().toString());
auxdataPreferences.put("OrbitFiles.sentinel1RESOrbitPath", "YOUR_PATH");
auxdataPreferences.put("OrbitFiles.sentinel1POEOrbitPath", "YOUR_PATH");
// ... Your code here ...

awesome, thanks.

I was checking the userDir and I was guessing that all the settings were properly in place, but as I see I needed to specify the Path for the orbits independently.

Config.instance().userDir()

was printing the value I set through VM argument

-Dsnap.userdir=C:\TEMP\mySnapDir

Thanks for your help

Hello,
I have updated the orbit files on the step website. Now the files should be up-to-date. Regarding the extension files or any other details about the QC site, I can not help.

Hello all,
I am using SNAP 6.0 and S1Toolbox 6.0.6. When applying orbit file, it gives me an error. But the error says, it couldn’t find the orbit file January 2000 but my SAR data is from 2017 and I downloaded manually the orbit files and replaced under 2000/1 and 2017/7(the correct one) but it is still same. I am not writing my java code. But could you help me about this issue?
Thanks.
Fikret

Maybe this and the following posts can help you:

If the dates are wrong it probably means that the S-1 product is defective (metadata issue?). You could try to download it again and see if it helps (there could be a reprocessed version).

The interesting thing is the program can find the orbit files of 20170718 but couldn’t find 20170730. It is also in the same ftp folder that I checked. It is hard to understand anyway I click don’t give error if orbit files couldn’t find so in this example program applying orbit file my master file but not slave.