Performance of SNAP used as a Java library on a server

development
#1

Hi to everyone. We are developing a experimental ESA supported project that brings some SNAP functionalities on-line in a web application.

The architecture has a html/JS client that calls REST API on a Java server. The Java server, when an operation is requested, shell executes another process, called Launcher, written in Java.

Launcher is the real core and is a Java main that links SNAP API and executes operations like ApplyOrbit, Terrain Correction, Radiometric Calibrate.

The software works well but there is a big performance issue. The same operations on our desktop PC, for example the ApplyOrbit, takes more or less 40’’.
On the server (16 cpu and 16Gb RAM), the same operation on the same file, takes up to 6’.
We tried to use the command line options like found in this forum. At the moment the command to run the launcher is:

java -Xmx11264m -Xms256m -XX:+AggressiveOpts -Xverify:none -Dsun.awt.nopixfmt=true -Dsun.java2d.dpiaware=false -Dsnap.parallelism=8 -jar /usr/lib/wasdi/launcher/launcher.jar -operation APPLYORBIT -parameter /usr/lib/wasdi/params/eeb2f11e-3160-430a-972d-fbbdafdc1138

In the code JAI is initialized using:
JAI.getDefaultInstance().getTileScheduler().setParallelism(Runtime.getRuntime().availableProcessors());
MemUtils.configureJaiTileCache();

But at the moment this does not help at all.
Any suggestion? How can we configure the launcher to be optimized ?

Thank you very much for your support.

#2

I would suggest not to call MemUtils. In bypasses the SNAP JAI initialisation. And thus you get a different configuration.
Beside this I see you use the snap.parallelism property in the launch command but set the parallelism to Runtime.getRuntime().availableProcessors(). This way the property has no effect.

I would suggest that you call:
SystemUtils.init3rdPartyLibs(null).
This will init JAI and GeoTools. The property snap.parallelism will be used and if you use the property snap.jai.tileCacheSize (in MB) also the cache size is configured.

Also it might be good to start the SNAP engine by calling:
Engine.start(false)
This will initialise some additional code. But maybe you won’t need it.

#3

Thank you very muich marpet! I’m going to try and let you know the results I think today.
Thank again for quick answer.

#4

Dear Marpet, we are making some test and at the moment still not solved.

I have to add an info: the server is a Linux Machine (Ubuntu: Linux version 4.4.0-57-generic (buildd@lgw01-54) (gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.4) ) #78-Ubuntu SMP Fri Dec 9 23:50:32 UTC 2016)

At the moment the command line is:
java -Xmx11264m -Xms1024m -XX:+AggressiveOpts -Xverify:none -Dsun.awt.nopixfmt=true -Dsun.java2d.dpiaware=false -Dsnap.parallelism=8 -jar /usr/lib/wasdi/launcher/launcher.jar -operation APPLYORBIT -parameter /usr/lib/wasdi/params/9513b305-f519-4833-b00c-d5a3112d47a6

The Constructor is now:

        //JAI.getDefaultInstance().getTileScheduler().setParallelism(Runtime.getRuntime().availableProcessors());
        //MemUtils.configureJaiTileCache();
        
        SystemUtils.init3rdPartyLibs(null);
        Engine.start(false);

The problem may be that we see in the log file:
2017-05-24 15:33:53 DEBUG LauncherMain:106 - Launcher Main Start
SEVERE: org.esa.s2tbx.dataio.gdal.GDALInstaller: The GDAL library is available only on Windows operation system.

And the operation is very very slow.
We are trying to investigate. Any help is welcome :slight_smile:

#5

The message regarding GDAL is false alert. You can ignore it.

Regarding the slowness. I think it is worth to add -Dsnap.jai.tileCacheSize=6000 as command line property. You can even go higher and set 8000.

#6

The problem with GDAL was a problem with the versioning of the SNAP code. Now is solved updating the Maven project.
Now the system run better.

To summarize we are using (as suggested by marpet):
.SNAP 6.0.0-SNAPSHOT
.REMOVED the reference to JAI and MemUtils
. Added in the constructor of our code:
SystemUtils.init3rdPartyLibs(null);
Engine.start(false);
.Running the JAR with this parameters:
java -Xmx11264m -Xms1024m -XX:+AggressiveOpts -Xverify:none -Dsun.awt.nopixfmt=true -Dsun.java2d.dpiaware=false -Dsnap.parallelism=8

We now will also try to add also
-Dsnap.jai.tileCacheSize=6000 or 8000

If possible we are planning to run also the benchmark on the server to have a comparable measure.
Thank you again very much for your help.