I am using Graph builder to process a time series of 60 Sentinel-1 images.
My preprocessing step includes the TOPSAR-Split and Apply-orbit-File for each image of the time series.
I have got the warning for downloading orbit files: WARNING: org.esa.s1tbx.io.orbits.sentinel1.StepAuxdataScraper: Unable to connect to http://step.esa.int/auxdata/orbits/Sentinel-1/POEORB/S1A/2017/07/: connect timed out WARNING: org.esa.s1tbx.io.orbits.sentinel1.StepAuxdataScraper: Unable to connect to http://step.esa.int/auxdata/orbits/Sentinel-1/POEORB/S1A/2017/08/: connect timed out WARNING: org.esa.s1tbx.sar.gpf.orbits.ApplyOrbitFileOp: No valid orbit file found for 24-JUL-2017 05:50:39.974600 Orbit files may be downloaded from https://qc.sentinel1.eo.esa.int/ and placed in /home/le/.snap/auxdata/Orbits/Sentinel-1/POEORB/S1A/2017/07 WARNING: org.esa.s1tbx.io.orbits.sentinel1.StepAuxdataScraper: Unable to connect to http://step.esa.int/auxdata/orbits/Sentinel-1/RESORB/S1A/2017/07/: connect timed out WARNING: org.esa.s1tbx.io.orbits.sentinel1.StepAuxdataScraper: Unable to connect to http://step.esa.int/auxdata/orbits/Sentinel-1/RESORB/S1A/2017/08/: connect timed out WARNING: org.esa.s1tbx.sar.gpf.orbits.ApplyOrbitFileOp: ApplyOrbit ignoring error and continuing: org.esa.snap.core.gpf.OperatorException: No valid orbit file found for 24-JUL-2017 05:50:39.974600 Orbit files may be downloaded from https://qc.sentinel1.eo.esa.int/ and placed in /home/le/.snap/auxdata/Orbits/Sentinel-1/POEORB/S1A/2017/07 …10%…20%…30%…40%…50%…60%…70%…80%…90% done.
There was no problem with my internet connection. Then it took 10 minutes for downloading the orbit file for each image, and then quickly finished the processing chain. Whilst, it just takes 30 seconds with manual step-by-step. Is it possible to have an option that allows the use of downloaded orbit file for Apply-Orbit-File step? Or does anyone have an solution for this?
My next step is to generate interferogram with the following graph:
I used the external DEM option in this processing chain, and I downloaded and merged the SRTM 1sec HGT DEMs from http://step.esa.int/auxdata/dem/SRTMGL1/ that cover my study area. But I also got the problem of execution time. When using GPT command line for my Graph, it took 2 hours to calculate the interferogram for each pairs of images (one swath of each scene). I tried to do manually the same steps of the processing chain and it took totally 20 – 25 minutes. I use a computer with Intel® Xeon® Silver 4110 CPU @ 2.10GHz × 32; 62.6 GB RAM, Ubuntu 18.04.1 LTS.
Is it normal when it takes much more time to process data by using graph builder than the manual operations? How to solve this problem? Thank you for your help!
Please correct me if I’m wrong but I think that if you already have the orb files, it doesn’t download them.
Concerning your overall problem, I had something similar times ago. It was solved by either editing the gpt.vmoptions file. you can see it in ~/snap/bin/
With your 64Go of RAM, you should have something like that :
# Enter one VM parameter per line
# For example, to adjust the maximum memory usage to 512 MB, uncomment the following line:
# -Xmx512m
# To include another file, uncomment the following line:
# -include-options [path to other .vmoption file]
-Xmx43G
You can also also change it using the SNAP-Configuration-Optimiser
Thank you for your answer, @qglaude
I will try to modify the gpt.vmoptions file.
I haven’t downloaded the orbit files yet. But I wonder if we can do the same thing like the DEM case, with the option for using external DEM, then we can use the downloaded DEM. In the Apply-Orbit-File step, we just have two options: “Sentinel Precise (Auto Download)”; and “Sentinel Restituted (Auto Download)”, so I don’t know if we can use downloaded orbit files?
We can see in the SNAP warning that: Orbit files may be downloaded from https://qc.sentinel1.eo.esa.int/ and placed in /home/le/.snap/auxdata/Orbits/Sentinel-1/POEORB/S1A/2017/07
So that we can download and store orbit files in directories for each month? How to choose the right orbit file for a specific acquisition?
When you download orbit files, it’s not for a specific acquisition but for a specific date. You have one orbit files for each day for each satellite (i.e. 2 orbits files per day).
If you want precise orbits for one day, you can simply download it from there https://qc.sentinel1.eo.esa.int/aux_poeorb/ and put it in the directory you mentioned. Note that that’s what SNAP is supposed to do. Maybe a firewall or something is blocking SNAP ? I’m not an expert in these things though.
If you still want to manually use precise orbits, you probably know that precise orbits are only available after a time lag of about 2 weeks.
My gpt.vmoptions file is already the same as what you mentioned:
I also changed the cache size for destop application (on Tools / Options) from 1024MB to 44032MB, but I got the error: Cannot construct DataBuffer. Then I changed this value into 33024MB. However, nothing has changed for the execution time when working with gpt command line. It still takes along time to process my graph.
I found your topic that you created about this issue, and your problem has been solved. How did you set your parameters?
Dear Mr. @qglaude,
many thanks for sharing your knowledge. Checking the suggested source https://qc.sentinel1.eo.esa.int/aux_poeorb/ , I have just noticed that are available several (not only one) orbit files per day.
BR
