Run gpt from SNAP built from source

[thho]

Hi @marpet and @MartinoF, I hope you can help me.

In this post

Build SNAP from github source with maven 3.6

Hello, ATM I try to simply: clone the snap engine, desktop and s1tbx repos from github checkout to a specific commit git clone https://github.com/senbox-org/snap-engine.git cd snap-engine git checkout 65c1230f35 cd .. git clone https://github.com/senbox-org/snap-desktop.git cd snap-desktop git checkout 36fbe009 cd .. git clone https://github.com/senbox-org/s1tbx.git cd s1tbx git checkout 63f5ac39e and compile with maven but it fails as soon as I try compiling the engine repo c…

@MartinoF helped me out and I use this docker file Dockerfile (1.1 KB)

FROM ubuntu:latest
# TODO: use alpine
#COPY . /sbas_docker
RUN apt update
RUN apt upgrade -y
RUN apt install openjdk-8-jdk openjdk-8-demo openjdk-8-doc openjdk-8-jre-headless openjdk-8-source -y
RUN apt install maven -y
RUN apt install git -y

RUN mkdir ~/snap2 && \
    cd ~/snap2 && \
    git clone https://github.com/senbox-org/snap-engine.git && \
    git clone https://github.com/senbox-org/snap-desktop.git && \
    git clone https://github.com/senbox-org/s1tbx.git && \
    cd snap-engine  && \
    git checkout b0781cd1f0 && \
    cd ../snap-desktop && \
    git checkout f07be09c && \
    cd ../s1tbx && \
    git checkout 2a1b260f4
    
#RUN mkdir ~/snap2/old
#RUN mv ~/snap2/s1tbx/s1tbx-op-insar/src/main/java/org/esa/s1tbx/insar/gpf/InterferogramOp.java ~/snap2/old
#RUN cp /sbas_docker/InterferogramOp.java ~/snap2/s1tbx/s1tbx-op-insar/src/main/java/org/esa/s1tbx/insar/gpf/InterferogramOp.java

RUN mvn clean install --file ~/snap2/snap-engine/ -DskipTests=true
RUN mvn clean install --file ~/snap2/snap-desktop/ -DskipTests=true
RUN mvn clean install --file ~/snap2/s1tbx/ -DskipTests=true

to built a custom SNAP where I exchange a specific Java Operator in the S1TBX (for simplicity I commented it out in the docker). In order to run it, I use gpt and this is my issue, because I can not use my old approach to call the gpt tool from the custom SNAP built from source.

This is how I did it in SNAP 6.x and it worked maybe half a year ago:

#!/bin/bash
# enable next line for debugging purpose
# set -x 
############################################
# Main processing
############################################

snap2=$1#path to the github clones
projectDirectory="$PWD"
graphXmlPath="home/thho/test_io.xml"#just a simple read write graph

#prepare to call the gpt from the costum SNAP version
callgpt="java -Dinstall4j.jvmDir=/usr/lib/jvm/java-8-openjdk-amd64/jre -Dexe4j.moduleName=$snap2/snap-desktop/snap-application/target/snap/bin/gpt -Dsnap.mainClass=org.esa.snap.core.gpf.main.GPT -Dsnap.home=$snap2/snap-desktop/snap-application/target/snap/bin//.. -Dsnap.extraClusters=$snap2/s1tbx/s1tbx-kit/target/netbeans_clusters/s1tbx -Djava.awt.headless=true -Dinstall4j.noLoggingFix=true -Dinstall4j.launcherId=781 -Dinstall4j.swt=false -Di4jv=0 -Di4jv=0 -Di4jv=0 -Di4jv=0 -Di4jv=0 -Di4j.vpt=true -classpath $snap2/snap-desktop/snap-application/target/snap/.install4j/i4jruntime.jar:$snap2/snap-desktop/snap-application/target/snap/snap/modules/ext/org.esa.snap.snap-core/org-esa-snap/snap-runtime.jar org.esa.snap.runtime.Launcher"

#read and write example
${callgpt} ${graphXmlPath} -e
cd "${projectDirectory}"

and here my simple test graph, the paths after the file tags has to be changed…

<graph id="Graph">
<version>1.0</version>
<node id="Read">
<operator>Read</operator>
<sources/>
<parameters class="com.bc.ceres.binding.dom.XppDomElement">
<file>
/home/thho/kitchen/sbas_wf/SBAS/snap_prepro/stack_deb_rename/20160827_20160803.dim
</file>
</parameters>
</node>
<node id="Write">
<operator>Write</operator>
<sources>
<sourceProduct refid="Read"/>
</sources>
<parameters class="com.bc.ceres.binding.dom.XppDomElement">
<file>/home/thho/kitchen/test/target.dim</file>
<formatName>BEAM-DIMAP</formatName>
</parameters>
</node>
</graph>

Normally, this called the gpt from the custom built and executed the graph…but now I get this error (i do have internet for the container it runs in)

root@7316e1d72ff4:/data/sbas_wf# source SBAS/snap_sbas_wf/test_io_cstm.bash /root/snap2
+ source SBAS/snap_sbas_wf/test_io_cstm.bash /root/snap2
++ snap2=/root/snap2
++ projectDirectory=/data/sbas_wf
++ graphXmlPath=/data/sbas_wf/SBAS/snap_sbas_wf/test_io.xml
++ callgpt='java -Dinstall4j.jvmDir=/usr/lib/jvm/java-8-openjdk-amd64/jre -Dexe4j.moduleName=/root/snap2/snap-desktop/snap-application/target/snap/bin/gpt -Dsnap.mainClass=org.esa.snap.core.gpf.main.GPT -Dsnap.home=/root/snap2/snap-desktop/snap-application/target/snap/bin//.. -Dsnap.extraClusters=/root/snap2/s1tbx/s1tbx-kit/target/netbeans_clusters/s1tbx -Djava.awt.headless=true -Dinstall4j.noLoggingFix=true -Dinstall4j.launcherId=781 -Dinstall4j.swt=false -Di4jv=0 -Di4jv=0 -Di4jv=0 -Di4jv=0 -Di4jv=0 -Di4j.vpt=true -classpath /root/snap2/snap-desktop/snap-application/target/snap/.install4j/i4jruntime.jar:/root/snap2/snap-desktop/snap-application/target/snap/snap/modules/ext/org.esa.snap.snap-core/org-esa-snap/snap-runtime.jar org.esa.snap.runtime.Launcher'
++ java -Dinstall4j.jvmDir=/usr/lib/jvm/java-8-openjdk-amd64/jre -Dexe4j.moduleName=/root/snap2/snap-desktop/snap-application/target/snap/bin/gpt -Dsnap.mainClass=org.esa.snap.core.gpf.main.GPT -Dsnap.home=/root/snap2/snap-desktop/snap-application/target/snap/bin//.. -Dsnap.extraClusters=/root/snap2/s1tbx/s1tbx-kit/target/netbeans_clusters/s1tbx -Djava.awt.headless=true -Dinstall4j.noLoggingFix=true -Dinstall4j.launcherId=781 -Dinstall4j.swt=false -Di4jv=0 -Di4jv=0 -Di4jv=0 -Di4jv=0 -Di4jv=0 -Di4j.vpt=true -classpath /root/snap2/snap-desktop/snap-application/target/snap/.install4j/i4jruntime.jar:/root/snap2/snap-desktop/snap-application/target/snap/snap/modules/ext/org.esa.snap.snap-core/org-esa-snap/snap-runtime.jar org.esa.snap.runtime.Launcher /data/sbas_wf/SBAS/snap_sbas_wf/test_io.xml -e -x
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.esa.snap.core.util.EngineVersionCheckActivator: Please check regularly for new updates for the best SNAP experience.
Error: null
java.lang.ExceptionInInitializerError
        at java.net.Socket.setImpl(Socket.java:521)
        at java.net.Socket.<init>(Socket.java:86)
        at sun.net.NetworkClient.createSocket(NetworkClient.java:197)
        at sun.net.NetworkClient.doConnect(NetworkClient.java:162)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
        at sun.net.www.http.HttpClient.<init>(HttpClient.java:242)
        at sun.net.www.http.HttpClient.New(HttpClient.java:339)
        at sun.net.www.http.HttpClient.New(HttpClient.java:357)
        at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1226)
        at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1162)
        at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1056)
        at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:990)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1570)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
        at java.net.URL.openStream(URL.java:1068)
        at org.esa.snap.core.util.VersionChecker.getRemoteVersion(VersionChecker.java:138)
        at org.esa.snap.core.util.VersionChecker.checkForNewRelease(VersionChecker.java:94)
        at org.esa.snap.core.util.EngineVersionCheckActivator.start(EngineVersionCheckActivator.java:21)
        at org.esa.snap.runtime.Engine.informActivators(Engine.java:222)
        at org.esa.snap.runtime.Engine.lambda$start$0(Engine.java:121)
        at org.esa.snap.runtime.Engine.runClientCode(Engine.java:189)
        at org.esa.snap.runtime.Engine.start(Engine.java:121)
        at org.esa.snap.runtime.Engine.start(Engine.java:90)
        at org.esa.snap.runtime.Launcher.run(Launcher.java:51)
        at org.esa.snap.runtime.Launcher.main(Launcher.java:31)
Caused by: java.lang.NullPointerException
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1847)
        at java.lang.Runtime.loadLibrary0(Runtime.java:871)
        at java.lang.System.loadLibrary(System.java:1124)
        at java.net.AbstractPlainSocketImpl$1.run(AbstractPlainSocketImpl.java:84)
        at java.net.AbstractPlainSocketImpl$1.run(AbstractPlainSocketImpl.java:82)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.AbstractPlainSocketImpl.<clinit>(AbstractPlainSocketImpl.java:81)
        ... 26 more

When I found my solution, by emulating the gpt call in the bash script, some years ago, it already felt very cloncy and now it seems to not work anymore…However, I could not find a solution for the ltest version. Can you tell me how I can use gpt when SNAP was built from source in order to use my own Operators in the S1TBX or update the gpt call that points to the SNAP built from source?

Thank you
thho

[MartinoF]

Hi @thho,

Sorry for the delay, I personally use IntelliJ IDEA when developing and on the SNAP wiki page there are instructions to how run GPT from your current development enviroment:

https://senbox.atlassian.net/wiki/spaces/SNAP/pages/24051775/IntelliJ+IDEA

I think you can use the configuration to use it on other IDE (I partially ported it for VS Code) or even without it (I did a simple script to launch SNAP Desktop from my working directory at some point).

I hope this will help you out.

Best regards,
Martino

[thho]

Hi @MartinoF, no worries, your reply was fast enough

Thanks for the link, I know the page, still atm I try to make the processing chain as lean as possible…Therefore I do not use IntelliJ IDEA.

What I normally did is just a git clone and mvn clean install after that I was able to run the GPT by calling it directly. The string I emulate in order to do this was

"java -Dinstall4j.jvmDir=/usr/lib/jvm/java-8-openjdk-amd64/jre -Dexe4j.moduleName=$snap2/snap-desktop/snap-application/target/snap/bin/gpt -Dsnap.mainClass=org.esa.snap.core.gpf.main.GPT -Dsnap.home=$snap2/snap-desktop/snap-application/target/snap/bin//.. -Dsnap.extraClusters=$snap2/s1tbx/s1tbx-kit/target/netbeans_clusters/s1tbx -Djava.awt.headless=true -Dinstall4j.noLoggingFix=true -Dinstall4j.launcherId=781 -Dinstall4j.swt=false -Di4jv=0 -Di4jv=0 -Di4jv=0 -Di4jv=0 -Di4jv=0 -Di4j.vpt=true -classpath $snap2/snap-desktop/snap-application/target/snap/.install4j/i4jruntime.jar:$snap2/snap-desktop/snap-application/target/snap/snap/modules/ext/org.esa.snap.snap-core/org-esa-snap/snap-runtime.jar org.esa.snap.runtime.Launcher"

is similar to the VM options reported on the wiki, but there are some things that are in the string, which normally is built in the gpt executable, which are not part of the documentation in the wiki and I fear that are the problems…I will spent some time dirty hacking stuff into the string and do trail and error…When I come up with a solution I will report it here…If you have any more ideas to that, your help is highly welcome

Cheers Thorsten