Snaphu export does not generate a corrfile

Eugine · January 13, 2021, 9:38am

Hi, everyone,

I have a problem. By executing a graph (Coheristration/Interferogram/Snaphu export) on Linux VM with Snap 8 using a shell script, I get following files:

Phase_ifg_…snaphu.hdr
Phase_ifg_…snaphu.img
UnwPhase_ifg_…snaphu.hdr

But I don’t get any coherence files !

Executing the same graph on Windows, where I have Snap 7, gives me all needed files, including coh-files.

My goal is to automatize the process, so I would like to process my images on Linux.

What do you think could be a reason of the problem? I tried to search on ESA-forum, but I haven’t found anything related.

Thanks in advance,
Eugine

SambAB · January 8, 2021, 4:19am

The coherence file (header and image) are missing on the snaphu_export folder. When I continue the unwrapping process, an error occurs as shown in the screenshot. Is someone has the same experience and how did you fix it

ABraun · January 8, 2021, 7:27am

If the coherence is present before the export, you can copy its img and hdr files (inside the data folder) into the export folder and rename it as requested in the error message.

SambAB · January 8, 2021, 7:41am

thank you. I follow your advice and it is working

ABraun · January 8, 2021, 8:36am

very good, thank you for reporting.

SambAB · January 12, 2021, 4:24am

i am sorry to bother you again but I did a mistake confirming that it was working, the problem is still. It was working when I used previous data before i update to snap 8. the coherence files was included in the snaphu exported file. Finally i uninstall snap8 and install snap7. then when I use the snaphu_export, the coherence files appears and the unwrapping phase works. I think you need to fix this problem with snap8 version

ABraun · January 12, 2021, 9:05am

I just tested the SNAPHU export with SNAP 8 and the coherence is included on my side.

Test with TerraSAR-X data
grafik

Test with Sentinel-1 data
grafik

ABraun · January 13, 2021, 9:41am

I have merged the posts on this topic, so we can search for a solution together.

Do you both work with graph files (xml) to conduct the preprocessing and snaphu export?

Eugine · January 13, 2021, 9:48am

Thanks for merging the topics! Also nice to be reassured, that I am not alone with this problem.

I work with xml, yes, to conduct the preprocessing/snaphu export.

And during the Linux execution, I got messages like:

INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: GDAL 2.4.2 found on system. JNI driver will be used
INFO: org.esa.s1tbx.io.sentinel1.Sentinel1Level1Directory: Less than 8K RAM. Using FileCacheImageInputStream

But nothing heavier than that.

ABraun · January 13, 2021, 10:20am

I tried with my data, also got some GDAL warnings, but the files were created

grafik

Do you see a difference to your side regarding the syntax? Maybe you can also try if the GUI and the gpt lead to different results to narrow it down.

Eugine · January 13, 2021, 10:58am

I see quite many differences…

INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: GDAL 2.4.2 found on system. JNI driver will be used.
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Installed GDAL 2.4.2 set to be used by SNAP.
INFO: org.esa.snap.core.util.EngineVersionCheckActivator: Please check regularly for new updates for the best SNAP experience.
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Installed GDAL 2.4.2 set to be used by SNAP.
Executing processing graph
INFO: org.esa.s1tbx.io.sentinel1.Sentinel1Level1Directory: Less than 8K RAM. Using FileCacheImageInputStream
INFO: org.esa.s1tbx.io.sentinel1.Sentinel1Level1Directory: Less than 8K RAM. Using FileCacheImageInputStream
INFO: org.esa.s1tbx.io.sentinel1.Sentinel1Level1Directory: Less than 8K RAM. Using FileCacheImageInputStream
INFO: org.esa.s1tbx.io.sentinel1.Sentinel1Level1Directory: Less than 8K RAM. Using FileCacheImageInputStream
INFO: org.esa.s1tbx.io.sentinel1.Sentinel1Level1Directory: Less than 8K RAM. Using FileCacheImageInputStream
INFO: org.esa.s1tbx.io.sentinel1.Sentinel1Level1Directory: Less than 8K RAM. Using FileCacheImageInputStream
INFO: org.hsqldb.persist.Logger: dataFileCache open start
INFO: org.esa.s1tbx.io.sentinel1.Sentinel1Level1Directory: Less than 8K RAM. Using FileCacheImageInputStream
INFO: org.esa.s1tbx.io.sentinel1.Sentinel1Level1Directory: Less than 8K RAM. Using FileCacheImageInputStream
INFO: org.esa.s1tbx.io.sentinel1.Sentinel1Level1Directory: Less than 8K RAM. Using FileCacheImageInputStream
INFO: org.esa.s1tbx.io.sentinel1.Sentinel1Level1Directory: Less than 8K RAM. Using FileCacheImageInputStream
INFO: org.esa.s1tbx.io.sentinel1.Sentinel1Level1Directory: Less than 8K RAM. Using FileCacheImageInputStream
INFO: org.esa.s1tbx.io.sentinel1.Sentinel1Level1Directory: Less than 8K RAM. Using FileCacheImageInputStream
…10%…20%INFO: org.esa.s1tbx.sentinel1.gpf.SpectralDiversityOp: Shifts written to file: /home/evgeny/.snap/var/log/IW2_range_shifts.json
INFO: org.esa.s1tbx.sentinel1.gpf.SpectralDiversityOp: Estimating azimuth offset for blocks in overlap: 1/1
INFO: org.esa.s1tbx.sentinel1.gpf.SpectralDiversityOp: Shifts written to file: /home/evgeny/.snap/var/log/IW2_azimuth_shifts.json
…31%…42%…52%…62%…74%…84%… done.
– org.jblas INFO Deleting /tmp/jblas8536374862626687298/libquadmath-0.so
– org.jblas INFO Deleting /tmp/jblas8536374862626687298/libjblas.so
– org.jblas INFO Deleting /tmp/jblas8536374862626687298/libjblas_arch_flavor.so
– org.jblas INFO Deleting /tmp/jblas8536374862626687298/libgfortran-4.so
– org.jblas INFO Deleting /tmp/jblas8536374862626687298

Eugine · January 13, 2021, 11:03am

Just some background info. Bash-scripting SNAP 7 doesn’t work at all on Linux. Issues are related to Java (GC overhead limit exceeded).

I’m suspecting a RAM error. Could it be it? I have a Dell computer, and back on the days I had problems with switching it on. As a solution, I took out one of two RAM plates (it was a beginning of the first lockdown, I couldn’t do more).

Could it be, that the computer still thinks in a way, that it can operate on a full RAM, and fails either completely (as with SNAP7) or partially (as with SNAP 8)? But at the same time, SNAP7 works well on Windows.

P.S. I’m going to try GUI snap on Linux. May be it will clarify things.

ABraun · January 13, 2021, 12:35pm

yes, possible. If your amount of RAM has changed, you might check if the settings are still valid: You find instructions here

Eugine · January 15, 2021, 10:13am

So, I raised RAM on my VM from 6 to 12 GB.

And I also tested desktop versions of SNAP7 and SNAP8.

Basically, the same results.

Complete failure (GC overhead limit exceeded) for SNAP7 and no corrfiles created for SNAP8

gnwiii · January 15, 2021, 10:55am

You don’t mention if you changed the settings according to the link in @abraun’s Jan. 13th post. Can you show us the default_options line from <snap install dir>/etc/snap.conf?

panjibrotoisworo · February 25, 2021, 1:35pm

I’m also encountering this issue. I’m using a Linux VM too with Ubuntu 20.04 LTS with SNAP 8.

But I don’t have any errors about GC overhead limit exceeded. I’m using 65 GB of RAM and using 45 GB of cache.

ABraun · February 25, 2021, 1:45pm

in case you are using gpt make sure you include this

-PincludeCoherence=true (for gpt)
<includeCoherence>boolean</includeCoherence> (within a graph XML)

during the interferogram formation.

Eugine · February 25, 2021, 1:50pm

Hi,

Sorry for such a late response, was busy with other projects.

I have finally decided to kill the VM and install Linux on the disk partition (I used to work like this before, but on Dell it is harder to make it).

Snap 8 still doesn’t work, same errors.

Snap 7 works perfectly. I guess, on my VM the problem was a small amount of available RAM.

panjibrotoisworo · February 25, 2021, 5:19pm

I’m using the same XML graphs in my Linux VM (SNAP 8) and my Windows OS (SNAP 7) and includeCoherence is set to true. I have not observed any errors but there is still not coh file. For now I will just use SNAP 7.

zzl · March 1, 2021, 6:32am

Hello everybody,
I am using SNAP 8.0, RAM 12 GB, windows10, 64Bit OS.
First problem: After did snaphu, didn’t create the ( coh…_snaphu.hdr, coh…snaphu.img) files in my linked exported folder. So I read your discuss here then copy coh.file changed rename paste snaphu exported folder. Last running it …(snaphu -f snaphu.conf Phase_ifg_VV_10Feb2020_22Feb2020.snaphu.img 6185) on cmd line but not complete get to result. Please help to me how to fix it
Regards,
Zolo

.