In addition do mdelgado’s comment, please make sure that the DEM is projected in geographic coordinates (lat/lon).
thanks for the help!
ofc this was what i should have as name since its the exact name in the snap parameters aswell, clear!
the process runs now but is stuck on proscessing slave 1, its busy for 4 hours now and i have no clue how i can see the progress in the command line. is it supposed to print something when on 10 - 100%?
and @ABraun the DEM is 90m resolution in wgs 84 (official epsg code for world)
here is my project.conf:
project.conf (761 Bytes)
i am processing 12 slaves and 1 master from jun till oct of sentinel 1-a for an area in greenland.
my machine has 16 gb, 4 cores and 8 logical processors and runs windows 10.
- my processor atm runs between 30 and 40% so i think i could up the processing capabilities but i am not really sure.
xmx in snap is set to 12gb
the pyth env you see in the project conf file is my conda 2.7 environment with pathlib and the other packages installed with the correct dependencies.
thank you for your help!
EDIT: when i pressed enter 100 times for some reason the process furthered and the same error regarding the external dem is still there, therfore i will link the coreg.xml here:
coreg_ifg_computation.xml (5.8 KB)
EDIT2:
After editing al demnname and external dem name parameters in all cm graphs (also ifg2run and subset_ifg) the process is running correctly.
hope other noobs like me can use this information haha.
I am happy that you solve it.
Sorry but I have been the entire day travelling with no time for answering…
Good processing!
Hey guys,
Does anyone know of any hardware requirements of the stamps processing?
I have changed my old desktop in a linux box to run the processing and run into a memory error in matlab.
The linux box only has 4gb of ram so upgrading is the best to solve the problem. I am in doubt though whether its a good idea to just add 4gb or should i step it up and make it 2x 8gb?
I have a windows laptop running on an I7, SSD and 16GB of Ram but i am hesitant about using virtual machines for all the processing. Virtual machines seem tedious work around for me that only bring problems and a lot of SSD space requirements.
I am processing either 12 or 24 images from Sentinel-1 splitted into 1 subswath and 3 bursts.
Well…no updates would be required.
But again it depends of many factors. I run processing that needed about 32 GB but you can make it lighter but selecting more restrictive thresholds on the processing.
The more sugar, the sweeter …
Regarding VMs you can ask for it at the ESA Research and service support. They have a lot of experience in serving VMs for this purpose, even sometimes with access to Sentinel data on DIAS. Check it out.
to be more precise:
- my stamps processing memory error already occurred at step 1: stamps(1,1), i think i cant change any parameters her right?
- when you say you can make the processing lighter with restrictive thresholds do you only mean the parameters in the stamps processing within Matlab or also parameters i could change in the Matlab settings itself?
- are there specific parameters in stamps processing that could help me reduce the processing power needed you know off? if they are in the Snap-Stamps workflow documentation Gitlab I will find it for sure.
Hi Gijs,
Which kind of memory error you got on step1 ? Can you please show that again?
You can prior stamps in matlab, choose a amplitude dispersion threshold more restrictive which will limit the PS candidates initially selected and surely produce less PS points as results, but more stable ones.
You can do that while using the mt_prep_snap command
mt_prep_snap MASTER <PATH_TO_INSAR_FOLDER> amp_disp_thr patch_az patch_rg
In the manual they use 0.4 for amp_disp, but you can decrease it 0.35 or 0.3
This obviously has a direct impact on the number of output PS points
Once within matlab, you can restrict thresholds as weed_std or weed_max in step 4, and also do a merging (which reduces from full resolution to your decided resolution) decreasing as well the output PS points. Please check StaMPS manual for that. You can also use other thresholds in step 2 such as max_topo_error decreasing up to 5 meters or so.
In case you may need more suggestions, do not hesitate to ask.
Good processing!
1 Like
Hey,
Thank you for the tips! I will be on my linux pc tommorow and then i can send the exact error but it had to do with an out of memory error on line 261 of the script. That is where ps_initial_gamma is called.
I indeed used 0.4 as threshold during mt_prep_snap since i read the value should be between 0.4 and 0.42 and since i first want to see the basic results i used 4, if i still run into problems i will use 0.35 or 0.3 because i rather have as much ps points as possible in the end.
Once i get further in the matlab steps i can use the other suggestions you made.
Do you also know of any matlab setting that tells matlab to limit the ram usage and process slower? Because the error happens because matlab requests more than my max memory available but i cant see hie much memory matlab demands at this step.
Edit: since my 4gb of extra ram fixed the problem i have not been able to reproduce the error. best thing for other people with this kind of problems is process in multiple patches using the parameters for mt_prep_snap.
Hey,
i have been using this package happily for the last 2 weeks now and have a question/request.
for my descending data i have combined data from 2 path/frame combinations and therefore i have to split them separately since my AOI is in IW2 in 1 image and IW3 in another. by splitting these files separately using different directories i got this working but in the coreg_ifg_topsar.py step only the slaves with correct name (IW2 or IW3) thats in the project.conf file are being processed.
could i just run it 2 times with different parameters for the subset in project.conf or does this bring problems? I don’t think the package ever intend to not make using 2 path/frame combinations work since if it has data over your AOI it doenst matter.
would you be able to clarify? maybe this can improve the package.
I am happy to hear that you are happily using the snap2stamps scripts. They were release specially fo that, making people happy! 
Indeed this is one of the modification that we are trying to bring on the next release.
From my experience and with the current version of the scripts, I confirm that you can run as 2 different processing, each of them with different IWx. Even if they can be together on the same folders (prior stamps export) they will not mix, as the scripts search for specific IW specified on the project.conf
You can create as many project.conf as wish, just rename them and then you can call them for example:
python splitting_slaves.py project_IW2.conf
python splitting_slaves.py project_IW3.conf
I hope I have answered your question
1 Like
@mdelgado thank you for your answer!
as we both thought it would work i just tried it and it actually doesnt…
it gives the error that the same subswath is expected at the coreg_ifg_topsar phase.
so this means back-geocoding is only possible for images with the same subswath if i am correct. this means i cant choose 1 master to be the master for slaves of 2 different path/frames combinations. do you know any workaround?
error message is:
SNAP STDOUT:INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
Executing processing graph
INFO: org.hsqldb.persist.Logger: dataFileCache open start
Same sub-swath is expected.
done.
Error: [NodeId: Back-Geocoding] Same sub-swath is expected.
- note: the splitting step is indeed no problem.
Well, it actually works if you use it properly.
You should have also the master image splitted for both IWs to work with.
In the sense, you can do 2 parallell processing over 2 AOIs from the same S1 slice, located in different IWs.
I know that works, just because I have done it already time ago.
Obviously you can only coregister images covering same area and same track, and hence being in the same subswath.
yeah but this means that 2 overlapping tiles but not the same path/frame combinations can not be used right? because the AOI is in a different subswath between the 2 images.
it is over the same area but just not in the same subswath
@mdelgado
Yes, this is always a post-processing task.
It is not possible to merge subswaths IWs from different tracks, even if next or spatially overlapping, because a series of changes on incident angles, acquisition times, etc… the software cannot handle that on the fly.
However, you can do that with the PSI results, but removing any phase shift on the overlapped areas (if any) using always the same ref point (better to select it on the overlapping area). Search some studies on literature, as they should be…
Sorry but snap2stapms does not help (at least for now) on this specific subject
Thank you @mdelgado for the clarification, it is back to the drawing board then i guess.
I will just process the 2 path/frame combinations seperately with their own respective masters and start reading for a tool to combine it after stamps processing. I am also reading upon combining ascending and descending data to get true vertical displacement. If I get any step by step instructions i will share with the forum again.
Goodluck!
1 Like
dear @mdelgado,
I read the paper on the snap2stamps package and had a few questions.
A lot of people see Terrain correction as a big part in reducing error, am i correct to say that this is in the topophase removal or in the stamps step 5 (correction look angle error)? if not, how could i manage to still do terrain correction with the final ps results, any papers on this topic?
i am trying to combine my ps results for both the ascending and descending data and read in your paper that you used the vector data and compared the points closest together. did you do this in ArcGIS or QGIS using the function with angles that can be fund in multiple papers or did you make use of special software?
i read you made use of a gps station(GCP) as reference point for the combination of the data. i sadly have no gps station in my area and am unable to find any information about using or setting a reference point in the StaMPS manual, should i have founf info there?
sorry for all these questions, i hope you can clarify some things for me. thank you for your help!
1 Like
StaMPS provides longitude latitude information for each resulting PS point. No need to do anything else than read the ps2.mat in which you may find the matrix lonlat which contains that information in WGS84. Remember that StaMPS does not correct the coordinates after estimating the PS height, so you may find some (small)shifts on the coordinates.
Terrain Correction is not TopoPhase Removal. Please understand that the first is for geocoding the radar data and the second for removing the topographic-related phase in the interferogram which is related to the Bperp between master and slave images.
I use the formulas found on my paper,which is referenced (you can search for it) and I programmed it into Matlab. At the moment I found not public software to do that. You need information on incident and heading angle (also provided by StaMPS).
With StaMPS you may select or not a reference point, but I suggest you select the same point for ascending and descending prior decomposing. You can choose some stable/rocky area where you know that at least there is no deformation (or very small and correct for it)
Using getparm you see the different possibilities to define a reference point/area using ref_centre_lonlat, ref_radius for example, you can define the reference point as a circular area.
I hope this helps
3 Likes
thank you very much!
i am really onlyt familiar with python but will see if i can construct a script to compare the data for ascending and descending data based on the lon and lat in the .mat file
so terrain correction is indeed done but then in step 5 and step 7 of stamps right? thank you for the clarification.
thank you for the tips on the reference points.
have a good day!
1 Like
No. Step 5 and Step 7 corrects errors in the phase and estimates the PS height (and other things), but does not update the coordinate information that StaMPS should have defined in variables created during the mt_prep_snap . That is why you need latitude and longitude to be exported from SNAP towards StaMPS.
1 Like
Hey,
Would snap2stamps be able to deliver the coherence at the ifg formation? Just change in the graph - add chorence band “yes” .
I know this is not necessary for StaMPS processing but could be handy for visually deciding whether to drop images at step 7 of stamps.