Snap2stamps package: a free tool to automate the SNAP-StaMPS Workflow

and you means i do not need to replace the two files i get from the forum?

In the github package you have everything you need.
In the forum you may find several versions of those files, but within the github package you get them updated.

I hope this helps

1 Like

ok,maybe there should a new text like"PS Using Manual_3.1" more exactly

Sorry but I dot not get it.
New StaMPS version is 4.1b and it seems well organised to me, including the User Manual.

Enjoy!

1 Like

thanks alot

1 Like

Thank you for the work on this script!
I was just going to write my own one from scratch but this is so much better.
I think in the future I would like to remove the .xml graphs and instead just use python + snappy scripts to make it nice and smooth.

Can you please help with the question in the end of this long post?

I have successfully ran snap2stamps in a ubuntu docker container on a server as follows…
First the master output naming convention caught me out.
For now you need to run it in GUI (split + orbit) with more than one burst and name the output file the same as the S1 product with a different suffix.
I think you should just make it part of processing.
Attached is an .xml file you could add, it takes a WKT polygon as input so it will extract only the bursts that overlap the polygon. master-split.xml (1.0 KB).
Run this with an input master scene and a wkt text file as follows (cat just reads the wkt file which looks like: POLYGON((18.16 -32.88,18.53 -32.88,18.53 -33.11,18.16 -33.11,18.16 -32.88)) get it from wkt online)

gpt ./master-split.xml -t “./S1A_XXXX_orb” -PsourceFile="./S1A_XXXX.zip" -Pwkt="$(cat ./aoi.wkt)"

I run the entire snap2stamps with a bash script wrapper as follows - I think snap2stamps should also do this. It can source the wkt from the project.conf file.

#! /bin/bash
gpt /home/scripts/master-split.xml -t gpt ./master-split.xml -t “./S1A_XXXX_orb” -PsourceFile="./S1A_XXXX.zip" -Pwkt="$(cat ./aoi.wkt)"
python ./slaves_prep.py ./project.conf
python ./splitting_slaves.py /.project.conf
python ./coreg_ifg_topsar.py /.project.conf
python ./stamps_export.py /.project.conf

I run all of this with a snap dockerfile that I pulled with:

FROM mrmoor/esa-snap-with-python:latest

I also had to install pathlib in the dockerfile with:

RUN apt-get update && apt-get install -y git python-pathlib

I also pull snap2stamps with:

RUN mkdir -p /home/software/ && cd /home/software/ &&
git clone https://github.com/mdelgadoblasco/snap2stamps.git &&
chmod +x /home/software/*

In the docker I then run an entrypoint script with this configure snappy:

echo “-Xmx64G” > /usr/local/snap/bin/gpt.vmoptions
echo “snap.parallelism = 16” >> /usr/local/snap/etc/snap.properties
/usr/local/snap/bin/snappy-conf /usr/bin/python
/usr/bin/python /root/.snap/snap-python/snappy/setup.py install

My problem is with the mt_prep_snap script. I use the one that comes with Stamps.
I am attempting to run Stamps with Octave instead of Matlab. Wouldn’t this be amazing?
So I am using Octave v4.4.1 in a docker with the csh shell and I change this line in mt_prep_snap from
matlab -nojvm -nosplash -nodisplay < $STAMPS/matlab/ps_parms_in_itial.m > ps_parms_initial.log
to
octave < $STAMPS/matlab/ps_parms_in_itial.m > ps_parms_initial.log

I get the following output:
mt_prep_snap 20170730 /home/in/131/INSAR_20170730 0.4

mt_prep_snap Andy Hooper, August 2017

Amplitude Dispersion Threshold: 0.4
Processing 1 patch(es) in range and 1 in azimuth

octave: X11 DISPLAY environment variable not set
octave: disabling GUI features
opening /home/in/131/INSAR_20170730/rslc/20150331.rslc…
Segmentation fault (core dumped)
9162
1472
mt_extract_cands Andy Hooper, Jan 2007

Patch: PATCH_1
selpsc_patch /home/in/131/INSAR_20170730/selpsc.in patch.in pscands.1.ij pscands.1.da mean_amp.flt f 1
file name for zero amplitude PS: pscands.1.ij0
dispersion threshold = 0.4
width = 9162
number of amplitude files = 0
Segmentation fault (core dumped)

psclonlat /home/in/131/INSAR_20170730/psclonlat.in pscands.1.ij pscands.1.ll
opening pscands.1.ij…
Error opening file pscands.1.ij

pscdem /home/in/131/INSAR_20170730/pscdem.in pscands.1.ij pscands.1.hgt
opening pscands.1.ij…
pscdem: Error opening file pscands.1.ij

pscphase /home/in/131/INSAR_20170730/pscphase.in pscands.1.ij pscands.1.ph
opening pscands.1.ij…
Error opening file pscands.1.ij

So the main error is the Segmentation fault when opening the first scene. The scene looks fine. Wonder if anyone has any ideas? Is this more a question for MAINSAR?

In terms of snap2stamps. Why does it create target.dim and target.data files in the PROJECT directory? I see the data are interferograms, is this then the output before export to Stamps? It would be nice if this had a better filename and output directory.

1 Like

Hi @andretheronsa,

Indeed your suggestions regarding snap2stamps are points in which I wanted to update for next release.
Regarding the data on the project directory… call target.dim I think it is a bug of the StaMPS export operator which I will handle (removing it after exporting) on the next version of the scripts.

I have one question regarding your master processing, is that method extracting only the full burst matching (intersecting) with the WKT polygon? In fact the polygon in WKT can be handle directly on my scripts (as done for the coregistration/interferogram step) but at the time I have created I had the doubt that it could be extracted the full burst (otherwise you get issues on the coregistration step) using a polygon.

I would love to try your approach for the master processing and include it on next version (indeed it is a big improvement in this regard). So thank you very much for your suggestion.

Regarding octave instead of maltab, that represents a big challenge in fact! but the mt_prep file does not use matlab, but binaries which should be compiled once you get StaMPS package. Is it correct or I am wrong?

Let me know, probably we could discuss about improving the scripts functionalities…

1 Like

Thanks for your replies and work!

Noted about the target.dim

Yes, so far in my testing that method extracts the full bursts intersecting the WKT. So it does not clip the bursts and it also does not pull any bursts not in the WKT. So I think it works well. So yes, that polygon=(geosjon x,y etc.) approach in your other scripts should work for the master script too. If this is included in the next version it will simplify your code by totally eliminating the Master format confusion as well as making it automated.

An Octave-StAMPS thread somewhere would be nice - my institute is not interested in paying for Matlab.
Since this community is all about open source, and my PhD is all about open source PSI processing I am trying REALLY hard to get Octave to work with Stamps. I have earlier in the year managed to get to step 5 in Octave without needing to change too much. Just going over the scripts and fixing one or two lines in each script after googling the matlab/octave known inconsistencies. One important learning point was that there was a bug in versions prior to Octave v4.4.1 that resulted in some needed Stamps scripts to fail.

I THINK you are wrong about the binaries, unless I am doing something wrong?
I can read the mt_prep_snap script which is a csh script that calls a matlab function in line 146…

I would like to help where I can. I need this workflow automated and robust for my PhD in any case.

2 Likes

Thanks again for your comments.

I am eager to find anything that could work with StaMPS in non matlab environment, so let me know if you are thinking to distribute your code or needing somebody to discuss or test your code.

Regarding mt_prep, you are right, I did not see that in addition to calamp and mt_extract_cands, it also calls matlab for running the ps_parms_initial.m

Thanks for your comments and good luck with the PhD!

1 Like

maybe this is relevant to someone here:

If S1 SLC data is already extracted*, the first two scripts have to be modified a bit because they only search for zip files. I made some changes so they also work for unzipped S1 files:
@mdelgado If you don’t mind, I would share them here. But if you try to avoid several versions being around (as it was the case for the updated stamps scripts in here, surely caused some confusion), I can also provide them to you and you decide if they can be integrated somehow. Currently, they only work with manifest.safe files (not both zip and safe, which would be the premium solution, but require some conditional phrase).

*In my case, the S1 SLC data are processed on the RSS Cloud Toolbox which has direct access to the archived data via a folder connection (/eodata/Sentinel-1/SAR/SLC/). This allows fast transfer of the data but they are already unzipped.

1 Like

Actually I have also another version ( :wink: ) I have done while using the VMs provided by the ESA Research and Service Support within their RSS CloudToolbox service , but I wanted to include it on next release.
If you agree, I would prefer to disseminate that within official release, but you can share them in the meanwhile.

that’s even better :+1:
I now remember that you once contacted me about the Toolbox, right?

I am currently testing stamps 4.0b6.
mt_prep_stamps worked great and using it in the RSS Toolbox makes many things easier. Currently, I get an error at step 2 - do you have an idea what could be the reason?

My guess is the version… you should use the latest development version available in github (https://github.com/dbekaert/StaMPS ). As mentioned in the past, this solves issues during step1 that affects step2. And for your log seems exactly that.

Normally that happens when your ifg contains borders with No data or zeros… had you checked that?

Btw, probably it was me the one replying on for the VM, yes. :slight_smile:
The world is very small…

1 Like

thank you for the response.
Although I downloaded the latest release 4.1b (https://github.com/dbekaert/StaMPS/releases/tag/v4.1-beta) and correctly sourced it, it still says

STAMPS: ########################################
STAMPS: ####### StaMPS/MTI Version 4.0b6 #######
STAMPS: #######  Beta version, Jun 2018  #######
STAMPS: ########################################

I also saw no borders in the interferograms.

Edit: I might have had an old path in bashrc, I will have to check this.

Had you checked it already? And? Solve the step’s 2 error?

I accidently removed a folder and now the Virtual Machine doesn’t start any longer because the folder was called in my bashrc :joy:
I have contacted the RSS support team who can hopefully reset my bashrc file.

I removed the old StaMPS installation (also from my PATH), deleted all temporary files, and compiled StaMPS 4.1b from scratch, sourced it and ran mt_prep_snap again.

Still, I get this error at step 2

Is the version number okay? The temporary matlab path also points to the new folder.

Well… it does not show the version 4.1b so it seems that it is loading some old version still…

Let me check with my matlab to see the logging…
Indeed starts with:

addpath(’/application2/software/StaMPS-master/matlab/’,path)
stamps(1,1)

STAMPS: ########################################
STAMPS: ####### StaMPS/MTI Version 4.0b6 #######
STAMPS: ####### Beta version, Jun 2018 #######
STAMPS: ########################################

But please run stamps(1,1) again with the new scripts, as it is in step 1 where the software takes care of the issue that appears in step 2.

Try and let me know

thank you for the response and testing at your side.

I run everything from scratch, including mt_prep_snap and stamps(1,1) and it seems fine until this step.
I deleted all previous versions of stamps, as well as links. After sourcing the CONFIG file, the new scripts are callable from the shell
All data is correctly exported, the preparation shows no zero amplitude pixels and reasonable mean amplitudes, in MATLAB the ‘insar_processor’ is snap, the initial baselines make sense.
But the error remains.

I’ll try a different case area.

Seems to work now. My previous area was near the coast so the SRTM might have been insufficient.
I am at step 2 now and will report when I completed the processing with StaMPS v4.1b

1 Like