SNAP compiled from source to set OUTPUT_PHASE=true

Hi,

I’m trying to get the flat earth phase component from SNAP following what suggested here by @Monkey: Is there any way to get flat-earth phase of InSAR?

I would just need to set OUTPUT_PHASE=true in this file: snap_source\s1tbx-master\s1tbx-op-insar\src\main\java\org\esa\s1tbx\insar\gpf\InterferogramOp.java

@marpet Is it possible to set that variable without recompiling? That would be the best scenario.

About the compilation from source, I’m facing several problems:

  1. I was able to compile snap-engine, snap-desktop and s1tbx with Java 8, however guide says explicitly to use Java 11.
    1.1 I am anyway unable to run SNAP using IntelliJ IDEA, but I’ll look into this again once I will be able to compile with Java 11.

  2. Using Java 11

echo $JAVA_HOME
/usr/lib/jvm/java-11-openjdk-amd64/

2.1 snap-desktop and s1tbx compile fine
2.2 snap-engine fails with this error, due to missing com.sun.nio.zipfs

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.5.1:compile (default-compile) on project snap-engine-utilities: Compilation failure: Compilation failure: 
[ERROR] /home/user/snap-engine-9.0.6/snap-engine-utilities/src/main/java/org/esa/snap/engine_utilities/util/ZipFileSystemBuilder.java:[3,25] package com.sun.nio.zipfs does not exist
[ERROR] /home/user/snap-engine-9.0.6/snap-engine-utilities/src/main/java/org/esa/snap/engine_utilities/util/ZipFileSystemBuilder.java:[4,25] package com.sun.nio.zipfs does not exist
[ERROR] /home/user/snap-engine-9.0.6/snap-engine-utilities/src/main/java/org/esa/snap/engine_utilities/util/ZipFileSystemBuilder.java:[20,26] cannot find symbol
[ERROR]   symbol:   class ZipFileSystemProvider
[ERROR]   location: class org.esa.snap.engine_utilities.util.ZipFileSystemBuilder

Is the package missing due to the Java SDK I’m using? The guide says that any SDK would be fine. I also tried the Liberica JDK and I’m getting the same error.

Ok, I found some more info about this. I’m working with SNAP 9, which is still supposed to be built with Java 8. The docs changed recently: Page Comparison - How to build SNAP from sources (v.25 vs v.26) - SNAP - Confluence

It would be better to mention that until SNAP 9 it is necessary to use Java 8, otherwise people would get confused.

Update:

IntelliJ IDEA doesn’t like that my home folder contains the @ character. Moving the project under / solves this issue. So, finally the current status is a running SNAP 9.0.6 with these components compiled from sources:

  • snap-engine 9.0.6
  • snap-desktop 9.0.6
  • s1tbx 9.0.3

I’m able to read tiff files, but it doesn’t open zip files. What should I check?
SNAP opens them correctly if I’m using the official release, so it’s not a corrupted file problem.
It returns the error: No appropriate reader found

Ok, so the problem seems to be that the Sentinel-1 toolbox was not loaded, since I can’t see the radar menu: SNAP error "appropriate reader not found" - #4 by lveci

Hi @clausmichele

The marpet account is not active anymore.
I’m now using this account. See also: Farewell & Welcome SNAP Community - meta - STEP Forum (esa.int)

Recompiling is the only option to get the flat earth phase.

I think it would be good if @jun_lu or @lveci make it a user option. Now that it has been asked for the second time.

Yes, the documentation has already been updated for SNAP 10. You are right it should have been mentioned that there is a difference between both versions.
@dolaf Maybe you can add this?

Regarding the “reader not found message”, check again How to run and debug SNAP from an IDE / IntelliJ IDEA. Especially the clusters and patches parameter and that you are using : as path separator.

1 Like

@Marco_EOM thanks for the help despite not being anymore working for the SNAP project directly :slight_smile:
I managed to have also the Sentinel-1 toolbox working setting the paths correctly! Now I will try to get the flat earth phase with the GUI and later with GPT.

I’m now trying to use the compiled version via GPT following @Marco_EOM instructions here: How to setup GPT commandline from source code? - #7 by marpet

However, it complains that it can’t find the HDF5 library. I tried to include it with command line options like this, but it still doesn’t find it:

cat /snap_source/compiled/bin/gpt.sh 
#!/bin/sh
java -cp "../modules/*:../lib/*:../modules/lib/*" -Dsnap.mainClass=org.esa.snap.core.gpf.main.GPT -Dsnap.home="../" -Xmx20G org.esa.snap.runtime.Launcher -Dncsa.hdf.hdflib.HDFLibrary.hdflib=/mnt/snap_source/compiled/modules/lib/amd64/libjhdf.so -Dncsa.hdf.hdf5lib.H5.hdflib=/snap_source/compiled/modules/lib/amd64/libjhdf5.so "$@"

Error:

INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.esa.snap.core.util.EngineVersionCheckActivator: Please check regularly for new updates for the best SNAP experience.
java.lang.UnsatisfiedLinkError: no jhdf5 in java.library.path
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
        at java.lang.Runtime.loadLibrary0(Runtime.java:870)
        at java.lang.System.loadLibrary(System.java:1122)
        at ncsa.hdf.hdf5lib.H5.loadH5Lib(H5.java:339)
        at ncsa.hdf.hdf5lib.H5.<clinit>(H5.java:266)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:264)
        at org.esa.snap.dataio.hdf5.Hdf5ProductWriterPlugIn.loadHdf5Lib(Hdf5ProductWriterPlugIn.java:148)
        at org.esa.snap.dataio.hdf5.Hdf5ProductWriterPlugIn.<clinit>(Hdf5ProductWriterPlugIn.java:45)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at java.lang.Class.newInstance(Class.java:442)
        at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
        at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
        at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
        at org.esa.snap.core.util.ServiceLoader.loadServices(ServiceLoader.java:40)
        at org.esa.snap.core.dataio.ProductIOPlugInManager.<init>(ProductIOPlugInManager.java:209)
        at org.esa.snap.core.dataio.ProductIOPlugInManager$Holder.<clinit>(ProductIOPlugInManager.java:214)
        at org.esa.snap.core.dataio.ProductIOPlugInManager.getInstance(ProductIOPlugInManager.java:45)
        at org.esa.snap.core.gpf.main.CommandLineArgs.detectWriterFormat(CommandLineArgs.java:326)
        at org.esa.snap.core.gpf.main.CommandLineArgs.parseArgs(CommandLineArgs.java:191)
        at org.esa.snap.core.gpf.main.CommandLineArgs.parseArgs(CommandLineArgs.java:79)
        at org.esa.snap.core.gpf.main.CommandLineTool.run(CommandLineTool.java:112)
        at org.esa.snap.core.gpf.main.GPT.run(GPT.java:59)
        at org.esa.snap.core.gpf.main.GPT.main(GPT.java:37)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.esa.snap.runtime.Launcher.lambda$run$0(Launcher.java:55)
        at org.esa.snap.runtime.Engine.runClientCode(Engine.java:189)
        at org.esa.snap.runtime.Launcher.run(Launcher.java:51)
        at org.esa.snap.runtime.Launcher.main(Launcher.java:31)
WARNING: org.esa.snap.dataio.hdf5.Hdf5ProductWriterPlugIn: class org.esa.snap.dataio.hdf5.Hdf5ProductWriterPlugIn: HDF-5 library not available: class java.lang.UnsatisfiedLinkError: ncsa.hdf.hdf5lib.H5.H5dont_atexit()I
Executing processing graph
 done.
java.lang.NoClassDefFoundError: org/jblas/ComplexDouble
        at java.lang.Class.getDeclaredMethods0(Native Method)
        at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
        at java.lang.Class.getDeclaredMethod(Class.java:2128)
        at org.esa.snap.core.gpf.internal.OperatorContext.implementsMethod(OperatorContext.java:509)
        at org.esa.snap.core.gpf.internal.OperatorContext.isComputeTileMethodImplemented(OperatorContext.java:481)
        at org.esa.snap.core.gpf.internal.OperatorContext.<init>(OperatorContext.java:145)
        at org.esa.snap.core.gpf.Operator.<init>(Operator.java:96)
        at org.esa.s1tbx.insar.gpf.InterferogramOp.<init>(InterferogramOp.java:65)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at java.lang.Class.newInstance(Class.java:442)
        at org.esa.snap.core.gpf.OperatorSpi.createOperator(OperatorSpi.java:134)
        at org.esa.snap.core.gpf.graph.NodeContext.initOperator(NodeContext.java:132)
        at org.esa.snap.core.gpf.graph.NodeContext.<init>(NodeContext.java:51)
        at org.esa.snap.core.gpf.graph.GraphContext.<init>(GraphContext.java:80)
        at org.esa.snap.core.gpf.graph.GraphContext.<init>(GraphContext.java:58)
        at org.esa.snap.core.gpf.graph.GraphProcessor.executeGraph(GraphProcessor.java:127)
        at org.esa.snap.core.gpf.main.DefaultCommandLineContext.executeGraph(DefaultCommandLineContext.java:86)
        at org.esa.snap.core.gpf.main.CommandLineTool.executeGraph(CommandLineTool.java:547)
        at org.esa.snap.core.gpf.main.CommandLineTool.runGraph(CommandLineTool.java:391)
        at org.esa.snap.core.gpf.main.CommandLineTool.runGraphOrOperator(CommandLineTool.java:287)
        at org.esa.snap.core.gpf.main.CommandLineTool.run(CommandLineTool.java:188)
        at org.esa.snap.core.gpf.main.CommandLineTool.run(CommandLineTool.java:121)
        at org.esa.snap.core.gpf.main.GPT.run(GPT.java:59)
        at org.esa.snap.core.gpf.main.GPT.main(GPT.java:37)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.esa.snap.runtime.Launcher.lambda$run$0(Launcher.java:55)
        at org.esa.snap.runtime.Engine.runClientCode(Engine.java:189)
        at org.esa.snap.runtime.Launcher.run(Launcher.java:51)
        at org.esa.snap.runtime.Launcher.main(Launcher.java:31)
Caused by: java.lang.ClassNotFoundException: org.jblas.ComplexDouble
        at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ... 35 more

Error: org/jblas/ComplexDouble

I also tried a more naive approach, replacing the module org-esa-s1tbx-s1tbx-op-insar.jar in the installed SNAP with the new compiled one with OUTPUT_PHASE=true, but unfortunately it doesn’t change the “normal” behaviour of the Interferogram operator and no phase (band fep_) is obtained in the output.

Conclusion:

I managed to run GPT with the compiled SNAP in this way, which solved the HDF5 library error by including it in the java.library.path
.

#!/bin/sh
java -cp "../modules/*:../lib/*" -Djava.library.path=".:./modules/lib/amd64/" -Dsnap.mainClass=org.esa.snap.core.gpf.main.GPT -Dsnap.home="../" -Xmx20G org.esa.snap.runtime.Launcher "$@"

Anyway, after deleting the cached modules of the already installed SNAP and replacing the jar with OUTPUT_PHASE=true, makes it to work using directly the official GPT! :yum:

Long story short: if you need as output the flat-earth and topographic phase components, just compile the Sentinel-1 toolbox with OUTPUT_PHASE=true and replace only the org-esa-s1tbx-s1tbx-op-insar.jar under snap/s1tbx/modules.
No need to use IDE and worry about various configurations.

1 Like