SNAP Configuration in HPC

[saiplanner]

Is my Snap configuration done correctly in HPC or can I increase any parameter to reduce the process time? A simple interferogram generation for 1 image is taking almost 2 hrs which is not expected in HPC. What parameters I have to change or include more?

Thanking you,

[pratiman@master1 ~]$ gpt --diag
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
SEVERE: org.esa.s2tbx.dataio.gdal.activator.GDALDistributionInstaller: The environment variable LD_LIBRARY_PATH does not contain the current folder ‘.’. Its value is ‘/home/pratiman/java_8_131/jre1.8.0_131/lib:/home/pratiman/lis/lib/lib:/home/pratiman/intel/libs/lib:/opt/hdf5/lib/:/opt/gcc-4.8.2/lib/:/opt/gcc-4.8.2/lib64/:/lscratch/intel/impi/5.0.1.035/intel64/lib:/lscratch/intel/composer_xe_2015.0.090/compiler/lib/intel64:/lscratch/intel/composer_xe_2015.0.090/mpirt/lib/intel64:/lscratch/intel/composer_xe_2015.0.090/ipp/…/compiler/lib/intel64:/lscratch/intel/composer_xe_2015.0.090/ipp/lib/intel64:/lscratch/intel/composer_xe_2015.0.090/compiler/lib/intel64:/lscratch/intel/composer_xe_2015.0.090/mkl/lib/intel64:/lscratch/intel/composer_xe_2015.0.090/tbb/lib/intel64/gcc4.4:/lscratch/intel/composer_xe_2015.0.090/compiler/lib/intel64:/lscratch/intel/composer_xe_2015.0.090/mkl/lib/intel64:/lscratch/intel/composer_xe_2015.0.090/compiler/lib/intel64:/lscratch/intel/composer_xe_2015.0.090/mpirt/lib/intel64:/lscratch/intel/composer_xe_2015.0.090/ipp/…/compiler/lib/intel64:/lscratch/intel/composer_xe_2015.0.090/ipp/lib/intel64:/lscratch/intel/composer_xe_2015.0.090/compiler/lib/intel64:/lscratch/intel/composer_xe_2015.0.090/mkl/lib/intel64:/lscratch/intel/composer_xe_2015.0.090/tbb/lib/intel64/gcc4.4:/usr/local/munge/lib:/lscratch/PGI/linux86-64/17.1/lib:/lscratch/PGI/linux86-64/2017/mpi/openmpi-1.10.2/lib::/home/pratiman/local/lib’.
SNAP Release version 6.0
SNAP home: /home/pratiman/vvsk/snap/bin//…
SNAP debug: null
SNAP log level: null
Java home: /home/pratiman/vvsk/snap/jre
Java version: 1.8.0_102
Processors: 16
Max memory: 39.1 GB
Cache size: 1024.0 MB
Tile parallelism: 16
Tile size: 512 x 512 pixels

To configure your gpt memory usage:
Edit snap/bin/gpt.vmoptions

To configure your gpt cache size and parallelism:
Edit .snap/etc/snap.properties or gpt -c ${cachesize-in-GB}G -q ${parallelism}

[dsmilo]

Are you processing locally? Are you working off your C or local drive? If you are not, that could cause a big delay. Is SNAP up to date? Might want to run any outstanding updates. Check your memory and storage, make sure you have plenty for this type of processing. Also, what kind of data are you processing? X-Band will take a little longer than C-Band for example since there are more samples to process. However, it shouldn’t take 2 hours.

[mengdahl]

1GB of cache is way too little.

[saiplanner]

As I am working on HPC, it has both CPU’s and GPU’s as well. So, how much I should assign?

[mengdahl]

I do not know what the optimal size is but here it is stated one can go up to 75% of max memory allocated to the JVM:

Gpt and Snap performance parameters - exhaustive manual needed!

First, your questions are useful because many others will be interested in the answers. Many of your questions (the possible exception being #3) are in the category of “excercises left for the reader” or the parable “give a person a fish and that person is fed for a day, but teach a person to fish and that person has food for life”. In other words, the answers are less useful than knowing how to work then out for yourself. Some insight into the gtp command can be gained by just looking at th…

…so try with 20GB for example and see how that goes.