S1TBX fails to perform terrain correction/reprojection

Microwave Toolbox Problem Reports
#1

Dear S1TBX Developers,

I designed small processing chain using graph builder to automatize processing. It works in S1TBX in batch mode but when I use gpt tool the processing stops at terrain correction without any errors. In fact the return code from echo $? is 0.

To conclude gpt and batch mode give different results for the same input (at least for me…).

Thank you in advance for any kind of help,

Kind Regards,

Jan Musial

graph file: https://dl.dropboxusercontent.com/u/18374070/GRDH2Sigma.xml

gpt invoke command:

/opt/snap/bin/gpt /archiwum/s1-test/GRDH2Sigma.xml -e -q 8 -f GeoTIFF -t /tmp/Sentinel-1//Polkowo/GRDH2Sigma/2015/IW/GRDH/A29/A29_15/S1A_IW_GRDH_1SDV_20150628T161902_20150628T161927_006576_008C16_F4D4_Polkowo_GRDH2Sigma_A29_15_sigma.tif /mnt/opol-data/Sentinel-1/2015/IW/GRDH/A29/A29_15/S1A_IW_GRDH_1SDV_20150628T161902_20150628T161927_006576_008C16_F4D4.zip

gpt stdout:

Picked up _JAVA_OPTIONS: -XX:ParallelGCThreads=2 -Xms5G -Xmx6G
SLF4J: Failed to load class “org.slf4j.impl.StaticLoggerBinder”.
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
INFO: org.geotools.referencing.factory.epsg.ThreadedEpsgFactory: Setting the EPSG factory org.geotools.referencing.factory.epsg.DefaultFactory to a 1800000ms timeout
INFO: org.geotools.referencing.factory.epsg.ThreadedEpsgFactory: Setting the EPSG factory org.geotools.referencing.factory.epsg.ThreadedHsqlEpsgFactory to a 1800000ms timeout
INFO: org.geotools.referencing.factory.epsg.ThreadedHsqlEpsgFactory: Building new data source for org.geotools.referencing.factory.epsg.ThreadedHsqlEpsgFactory
INFO: org.geotools.referencing.factory.epsg.ThreadedHsqlEpsgFactory: Building backing store for org.geotools.referencing.factory.epsg.ThreadedHsqlEpsgFactory
INFO: org.geotools.referencing.factory.epsg.ThreadedEpsgFactory: Setting the EPSG factory org.geotools.referencing.factory.epsg.DefaultFactory to a 1800000ms timeout
INFO: org.geotools.referencing.factory.epsg.ThreadedEpsgFactory: Setting the EPSG factory org.geotools.referencing.factory.epsg.ThreadedHsqlEpsgFactory to a 1800000ms timeout
INFO: org.geotools.referencing.factory.epsg.ThreadedEpsgFactory: Setting the EPSG factory org.geotools.referencing.factory.epsg.DefaultFactory to a 1800000ms timeout
INFO: org.geotools.referencing.factory.epsg.ThreadedEpsgFactory: Setting the EPSG factory org.geotools.referencing.factory.epsg.ThreadedHsqlEpsgFactory to a 1800000ms timeout
INFO: org.geotools.referencing.factory.epsg.ThreadedEpsgFactory: Setting the EPSG factory org.geotools.referencing.factory.epsg.DefaultFactory to a 1800000ms timeout
INFO: org.geotools.referencing.factory.epsg.ThreadedEpsgFactory: Setting the EPSG factory org.geotools.referencing.factory.epsg.ThreadedHsqlEpsgFactory to a 1800000ms timeout

s1tbx info:

Product Version: SNAP 201411181905
Java: 1.8.0_45; Java HotSpot™ 64-Bit Server VM 25.45-b02
Runtime: Java™ SE Runtime Environment 1.8.0_45-b14
System: Linux version 3.13.0-57-generic running on amd64; UTF-8; en_gb (snap)

system info:

LSB Version: core-2.0-amd64:core-2.0-noarch:core-3.0-amd64:core-3.0-noarch:core-3.1-amd64:core-3.1-noarch:core-3.2-amd64:core-3.2-noarch:core-4.0-amd64:core-4.0-noarch:core-4.1-amd64:core-4.1-noarch:security-4.0-amd64:security-4.0-noarch:security-4.1-amd64:security-4.1-noarch
Distributor ID: Ubuntu
Description: Ubuntu 14.04.2 LTS
Release: 14.04
Codename: trusty

#2

I have the same problem with a graph which does the following:

Read -> Calibration -> Multi-Looking -> Terrain Flattening -> Terrain Correction -> Write

this gets stuck after 20% processing (in the interactive s1tbx). It never finished as a stand-alone gpt run (on a machine with 2x 6 cores and 148 Gb RAM!).

The part graphs:

Read -> Calibration -> Multi-Looking -> Terrain Flattening -> Write

Read -> Terrain Correction -> Write

in series work without problem. So, I’ll get what I want eventually, but would expect it to run in one go.

Guido

SNAP 2.0 Beta 05, ubuntu 14.04/64 bit, Java 1.8.0_60

#3

I’m not sure why it wouldn’t be completing however, you should never multilook or anything else that changes geometry/resolution before calibration or terrain flatten.
read -> Cal -> TF -> ML -> TC -> write

#4

Thanks, Luis, for this correction. However, things are not much better.

I can now do only

Read -> Cal -> TF -> write (4min32 on my machine)

(If I add -> ML after TF, the process is still not finished after 30 mins)

and

Read -> ML -> TC -> write (3min56)

I can do

read -> Cal -> ML -> TC -> write

in one graph and in 4min17

So, adding TF is not trivial, but also seems to block when both steps are combined.

Guido

#5

Why should one not be able to multilook before terrain flattening? Given that GRDH products with 10m postings cannot be compensated at “native” resolution using SRTM3 (with ~90m resolution), multilooking should be available as a way to expedite generation of a quick flattening result.

TF failed for a temperate latitude GRDH on our machine with 16GB. It completed on a machine with 32GB. It is not clear why more than 16GB should be necessary, even when ML is not allowed.

#6

My mistake. Calibration cannot currently occur after multilooking because the multilook operator does not update the calibation vectors (It updates the gecoding and abstracted metadata). However, the terrain flattening doesn’t need this and should be able to work after multilooking.
We will look into possibly updating the vectors accordingly in multilook.