Apply-Orbit-File thrugh corporative proxy

Hi there

First of all, congratulations for your great work

I am facing a problem. When trying to apply the orbit correction to a Sentinel-1 image trough gpt, the restituted orbit cannot be downloaded from https://qc.sentinel1.eo.esa.int. However, I have checked that, indeed, the orbit file exist.

As I am downloading it behind a corporative proxy, the main suspect was a configuration issue. However, I set the -Dhttps.proxyHost and -Dhttps.proxyPort (everywhere), and still nothing. Note that the proxy does not require user authentication.

After checking the differences with chrome and gpt (using wireshark), it seems that the connection to https://qc.sentinel1.eo.esa.int is done directly, not through the configured proxy. However, chrome was requesting the file through the proxy.

I could find the code where the connection is done, but I am not a java expert, I am sorry. However, let me know if I can provide you with more information.

Thanks in advance

Luis

Facing exactly the same problem, have you finally found a solution?

Well, we found several solutions, but not fancy ones:

  • Negociate with your company to open a dedicated connection (our solution).
  • Download manually the DEM into the Snap cache.
  • Configure a small VM with Squid as transparent proxy (we did not get it work, but we know about people that got it).
  • Get an external VPN and configure OpenVPN for handling all the problematic URLs. This is not recommended, as probably your company will be mad at you.

Without changin Snap code, as you can see, all we can get are workarounds

Have you identified a bug in the way SNAP handles proxies?

I am using my awful memory now, and I am not a Java expert. Therefore, maybe all that I am saying here is a nonsense. Sorry if it is the case.

When trying to get this to work, I got until the part that downloads the orbit files. After googling a little bit, I found that it is using a class that did not honour the system proxy settings. However, as I said before, I am not a java expert, so I decided to get the way of getting rid of the proxy itself. I could not either make further researchs and, sincerelly, after 3 months without response, I considered this was not an important problem to anybody but me, so well…

Kind regards

Luis

Still, the manual configuration should work? Can you find the name of the offending library?

SNAP has a large number of users and only very few hae had persistent proxy-issues. If there’s a bug in SNAP we will try to fix it of course!

Thank you very much for your answers and the solutions proposed.

I’m not a java expert neither (far from that), but the persons in charge of our network configuration tried hard to fix this issue without any solution with the manual configuration.
All the data flow is going through the proxy, with the exception of the orbit files for which the system is trying to connect directly to the server (which cannot be done in my company, and I imagine for many others).

Thanks again for your help and assistance,

Hervé

Thank you for the report. @lveci any ideas what’s going on with the orbit files?

Hello,
are there any news to this? I am facing the same problem. I managed to get the terrain-correction running, which needs to download the DEM, but no luck with the apply orbit file functionality.
Best,
Anna

We’re also experiencing problems trying to negotiate the proxy using gpt. I tried setting proxy variables in gpt.vmoptions and other tips I found searching these forums, but was not successful. My current workaround is to pre-download both the orbit files and SRTM DEMS, but this is not a good solution. It would be better if we could get gpt to honour a proxy if set.

Hello,
this problem is almost 1,5 years old and still no reaction or fixes for that.
We are facing the same problem (February 2021).
I am speaking in the name of all federal research instituitions in the EU, working behind a proxy.

We want to process a very large amount of Sentinel-1 data for the whole country.
We are using the python api (snappy) to get this job done.
Up to know, everything works except for the orbit file correction.

I ask the developers of SNAP to take this problem seriously and fix this bug so that we can run our scripts with snappy.

Can you clarify what the bug is? We obviously cannot make your firewall let SNAP connect to the orbit files repository. @lveci are the connections for orbits not handled the same manner as other connections?

I don’t know what happens in the code of SNAP.
I will try to explain the problem now, but I am not a computer scientist who uses the right vocabulary for this.

We work, as I said, behind a proxy that can be specified in Windows and also in most software packages.
This is also the case in the graphical interface of SNAP under Options, WWW (Figure 1).
There you can also test the connection and it works.

However, if you then want to perform the Orbit file correction, an error message appears (Figure 2).
So it seems that despite the proxy settings, the function does not use the information.

However, we do not use SNAP, but the PythonAPI Snappy, because we want to process mass data, i.e. thousands of Senttinel-1 scenes.
Here there must be a possibility to include the proxy.
The error message is the same here (Figure 3).

I had the same problem elsewhere and was able to solve it. When using the python package sentinelsat I got the same error message when the script tried to retrieve data from sentinel-hub.
The solution here was the following lines of code at the beginning of the script.

import os
proxy='http://proxy.julius-kuehn.de:8000'
os.environ['http_proxy'] = proxy
os.environ['HTTP_PROXY'] = proxy
os.environ['https_proxy'] = proxy
os.environ['HTTPS_PROXY'] = proxy

This solution probably doesn’t work for Snappy, because the package uses Java in the background, not Python. I also set the proxy in Java on my local machine, but without success. I now assume that SNAP brings its own Java environment to prevent problems with updates and not break dependencies.

Figure1:

Figure2:

Figure 3:

Hello? is there anyone who could answer to our problem?

An additinal question to this problem is:
When I try to get the orbit file, what address/port is used in the background.
I could try to negotiate a ssl tunnel with my institute.
This would be however a workaround but no proper solution.

Any help, please?

@lveci could you please take a look at this issue?

Any help, please?

It would also be helpful if someone could tell us that we are wrong and that the problem is not with the software. Maybe @lherreroc’s workaround 1 is the only solution possible? Maybe it can’t be solved on the software side at all? We don’t know. It would be great if someone could take a look again in this issue…

Hi florian,

did you find a solution to this problem? I’ve tried all the proposed solutions in the forums that i found since 2016 …but non of them …SNAP does not retrieve the orbits even if I download them manually SNAP doesn’t read them…

Any answer to this problem would be great.
Many thanks!

this might be related to a recent issue: Orbit file timeout (March 2021)
Hopefully fixed soon.

@ABraun @geo021 I guess we have different problems. My problem is, that my institution is working behind a proxy and it has very high restrictions regarding data exchange via internet. So, our problem in this post is something with ssl handshake…

My python script was working perfectly outside of my institution network.

The problem you are facing is maybe another one and according to @ABraun everyone has this issue, currently ?!

1 Like