Hi,
terrain flattening with the default srtmgl1 hgt option works fine, but whenever I specify an external dem, I get a nullpointer exception. I remember reading in some other post that the combination of regrid=true (default) + external dem is not supported, is that true?
Any tips on using an external dem with terrain flattening?
thanks!
the DEM must be projected in geographic coordinates (lat/lon) and stored as a GeoTiff.
And of course, it must coarsely match the area of your SAR image.
All that is true:
gdalinfo dem.tif
Driver: GTiff/GeoTIFF
Files: dem.tif
Size is 9976, 7864
Coordinate System is:
GEOGCS[“WGS 84”,
DATUM[“WGS_1984”,
SPHEROID[“WGS 84”,6378137,298.257223563,
AUTHORITY[“EPSG”,“7030”]],
AUTHORITY[“EPSG”,“6326”]],
PRIMEM[“Greenwich”,0],
UNIT[“degree”,0.0174532925199433],
AUTHORITY[“EPSG”,“4326”]]
Origin = (101.207916666703966,5.218472222214956)
Pixel Size = (0.000277777777778,-0.000277777777778)
Metadata:
AREA_OR_POINT=Area
Image Structure Metadata:
COMPRESSION=DEFLATE
INTERLEAVE=BAND
Corner Coordinates:
Upper Left ( 101.2079167, 5.2184722) (101d12’28.50"E, 5d13’ 6.50"N)
Lower Left ( 101.2079167, 3.0340278) (101d12’28.50"E, 3d 2’ 2.50"N)
Upper Right ( 103.9790278, 5.2184722) (103d58’44.50"E, 5d13’ 6.50"N)
Lower Right ( 103.9790278, 3.0340278) (103d58’44.50"E, 3d 2’ 2.50"N)
Center ( 102.5934722, 4.1262500) (102d35’36.50"E, 4d 7’34.50"N)
Band 1 Block=64x64 Type=Int16, ColorInterp=Gray
Min=-115.000 Max=2183.000
Minimum=-115.000, Maximum=2183.000, Mean=226.731, StdDev=328.554
NoData Value=-32768
Metadata:
STATISTICS_MAXIMUM=2183
STATISTICS_MEAN=226.73069839892
STATISTICS_MINIMUM=-115
STATISTICS_STDDEV=328.55389829068
And here is an image of the dem, with the S1 footprint in red:
Any other ideas? Has anyone actually succesfully done terrain-flattening with an external dem?
Your DEM should be fine then.
But are you sure you are working with the latest version of SNAP?
Watch out for this icon on the lower right 
or select Help > Check for updates
The re-gridding option is no longer available in the Terrain Flattening module. This is how it looks in the meanwhile.
I’m doing this purely command-line based, but I think I am fully updated.
For reference (filtered on s1tbx for size’ sake):
snap --nosplash --nogui --modules --list --refresh | grep -i s1tbx
org.esa.s1tbx.s1tbx.kit 6.0.6 Enabled
org.esa.s1tbx.s1tbx.commons 6.0.6 Enabled
org.esa.s1tbx.s1tbx.op.analysis.ui 6.0.6 Enabled
org.esa.s1tbx.s1tbx.op.feature.extraction 6.0.6 Enabled
org.esa.s1tbx.s1tbx.op.sar.processing 6.0.6 Enabled
org.esa.s1tbx.s1tbx.rcp 6.0.6 Enabled
org.esa.s1tbx.s1tbx.op.insar 6.0.6 Enabled
org.esa.s1tbx.s1tbx.op.calibration.ui 6.0.6 Enabled
org.esa.s1tbx.s1tbx.op.utilities.ui 6.0.6 Enabled
org.esa.s1tbx.s1tbx.op.ocean.ui 6.0.6 Enabled
org.esa.s1tbx.s1tbx.io.ephemeris 6.0.6 Enabled
org.esa.s1tbx.s1tbx.op.utilities 6.0.6 Enabled
org.esa.s1tbx.s1tbx.op.feature.extraction.ui 6.0.6 Enabled
org.esa.s1tbx.s1tbx.op.sar.processing.ui 6.0.6 Enabled
org.esa.s1tbx.s1tbx.op.sentinel1.ui 6.0.6 Enabled
org.esa.s1tbx.s1tbx.op.sentinel1 6.0.6 Enabled
org.esa.s1tbx.s1tbx.op.insar.ui 6.0.6 Enabled
org.esa.s1tbx.s1tbx.io 6.0.6 Enabled
org.esa.s1tbx.s1tbx.op.calibration 6.0.6 Enabled
However, gpt Terrain-Flattening -h still shows the reGridMethod option:
gpt Terrain-flattening -h INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters SEVERE: org.esa.s2tbx.dataio.gdal.activator.GDALDistributionInstaller: The environment variable LD_LIBRARY_PATH does not contain the current folder '.'. Its value is '/usr/local/lib'. Usage: gpt Terrain-flattening [options] Description: Terrain Flattening Source Options: -Ssource=<file> Sets source 'source' to <filepath>. This is a mandatory source. Parameter Options: -PadditionalOverlap=<double> The additional overlap percentage Valid interval is [0, 1]. Default value is '0.1'. -PdemName=<string> The digital elevation model. Default value is 'SRTM 1Sec HGT'. -PdemResamplingMethod=<string> Sets parameter 'demResamplingMethod' to <string>. Default value is 'BILINEAR_INTERPOLATION'. -PexternalDEMFile=<file> Sets parameter 'externalDEMFile' to <file>. -PexternalDEMNoDataValue=<double> Sets parameter 'externalDEMNoDataValue' to <double>. Default value is '0'. -PoutputSimulatedImage=<boolean> Sets parameter 'outputSimulatedImage' to <boolean>. Default value is 'false'. -PoversamplingMultiple=<double> The oversampling factor Valid interval is [1, 4]. Default value is '1.5'. -PreGridMethod=<boolean> Sets parameter 'reGridMethod' to <boolean>. Default value is 'true'. -PsourceBands=<string,string,string,...> The list of source bands. Graph XML Format: <graph id="someGraphId"> <version>1.0</version> <node id="someNodeId"> <operator>Terrain-Flattening</operator> <sources> <source>${source}</source> </sources> <parameters> <sourceBands>string,string,string,...</sourceBands> <demName>string</demName> <demResamplingMethod>string</demResamplingMethod> <externalDEMFile>file</externalDEMFile> <externalDEMNoDataValue>double</externalDEMNoDataValue> <outputSimulatedImage>boolean</outputSimulatedImage> <additionalOverlap>double</additionalOverlap> <oversamplingMultiple>double</oversamplingMultiple> <reGridMethod>boolean</reGridMethod> </parameters> </node> </graph>
Has anyone ever got this working? If so, could you please comment, so I know al my trials are not in vain…
Or maybe someone could just try, for validation’s sake…?
I’m still trying to get Terrain-Flattening working with an external dem. My dem is a geotiff, it matches the scene footprint with overlap, but I still get a nullpointer exception whatever I try.
Are there any more restrictions to the dem (format)? It does work with Terrain-Correction (with the same dem)…
I’m running this command:
gpt Terrain-Flattening -PexternalDEMFile=…/…/…/misc/srtmgl1_S1A_IW_GRDH_1SDV_20190106T225523_20190106T225548_025363_02CED2_79C7.tif -PexternalDEMNoDataValue=-32768 -t test_tf S1A_IW_GRDH_1SDV_20190106T225523_20190106T225548_025363_02CED2_79C7_AOF_TNR_CAL.dim
the dem tif in fact is just a mosaic of srtmgl1 1 arcsec data (for testing purposes).
I tried it yesterday (in the GUI) and experienced the same error. I don’t know what causes it though.
Thanks for testing and confirming, Andreas.
Any idea how to proceed here? We’d really like to get this working and be able to use a better and higher resolution dem for the terrain flattening. Is there a way to debug this, or to get a more informative backtrace? Should I file a bug in the s1tbx github?
Thanks,
Vincent.
the problem has been resolved in SNAP v7. I used a 10m DEM with WGS84 and integer heights and SNAP was able to use it in the Radiometric Terrain Flattening
It seems to work in some runs and make errors in others starting with
Problem reading DEM: null
…and ending …
Error: javax.imageio.IIOException: 16-bit samples are not supported for Horizontal differencing Predictor
So this error might be related to compressed DEM files or similar old errors…
The DEM file must be uncompressed TIFF. Your’s is COMPRESSION=DEFLATE.
Use gdal_translate to convert to an uncompressed tiff (the default)?
Side note: This issue with compressed TIFFs and DEMs originally popped up when I installed S2TBX years ago. With only S1TBX installed I believe I could use compressed TIFFs.
The funny detail is, that more than half of the single scenes still get terrain-flattening working, just some of the files from one day are failing. But I will test with an uncompressed dem … let’s see.
Still fails in the same way as with a compressed DEM. Something else is wrong!
Here is the output of the failing gpt run:
snap/bin/gpt conda/sen2mosaic/lib/python3.7/site-packages/sen1mosaic-0.2-py3.7.egg/sen1mosaic/…/cfg/3_terrain_correction_db.xml -x -Pinputfile=/data/incoming/sat/S1B_IW_GRDH_1SDV_20190811T154022_20190811T154050_017540_020FD1_5F6D_processed_cal_mtl_1t.dim -Poutputfile=/data/incoming/sat/S1_processed_20190811_154022_154050_017540_020FD1 -Pextent=500,500,11712,9576
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
SEVERE: org.esa.s2tbx.dataio.gdal.activator.GDALDistributionInstaller: The environment variable LD_LIBRARY_PATH is not set. It must contain the current folder ‘.’.
Executing processing graph
INFO: org.hsqldb.persist.Logger: dataFileCache open start
…java.lang.NullPointerException
java.lang.NullPointerException
java.lang.NullPointerException
java.lang.NullPointerException
java.lang.NullPointerException
90% done.
org.esa.snap.core.gpf.OperatorException: java.lang.NullPointerException
at org.esa.snap.core.gpf.graph.GraphProcessor$GPFImagingListener.errorOccurred(GraphProcessor.java:363)
at com.sun.media.jai.util.SunTileScheduler.sendExceptionToListener(SunTileScheduler.java:1646)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:921)
at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
at org.esa.snap.core.datamodel.Band.readRasterData(Band.java:311)
at org.esa.snap.core.dataio.ProductSubsetBuilder.readBandRasterDataRegion(ProductSubsetBuilder.java:328)
at org.esa.snap.core.dataio.ProductSubsetBuilder.readBandRasterDataImpl(ProductSubsetBuilder.java:297)
at org.esa.snap.core.dataio.AbstractProductReader.readBandRasterData(AbstractProductReader.java:250)
at org.esa.snap.core.gpf.common.SubsetOp.computeTile(SubsetOp.java:270)
at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.getPixelValue(RangeDopplerGeocodingOp.java:1229)
at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.computeTileStack(RangeDopplerGeocodingOp.java:1039)
at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeRect(OperatorImageTileStack.java:116)
at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeTile(OperatorImageTileStack.java:85)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
at org.esa.snap.raster.gpf.LinearTodBOp.computeTile(LinearTodBOp.java:116)
at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:73)
at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at com.sun.media.jai.util.RequestJob.compute(SunTileScheduler.java:247)
at com.sun.media.jai.util.WorkerThread.run(SunTileScheduler.java:468)
Caused by: org.esa.snap.core.gpf.OperatorException: java.lang.NullPointerException
at org.esa.snap.engine_utilities.gpf.OperatorUtils.catchOperatorException(OperatorUtils.java:432)
at org.esa.s1tbx.sar.gpf.geometric.TerrainFlatteningOp.computeTileStack(TerrainFlatteningOp.java:497)
at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeRect(OperatorImageTileStack.java:116)
at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeTile(OperatorImageTileStack.java:85)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
… 45 more
Caused by: org.esa.snap.core.gpf.OperatorException: java.lang.NullPointerException
at org.esa.snap.engine_utilities.gpf.OperatorUtils.catchOperatorException(OperatorUtils.java:432)
at org.esa.s1tbx.sar.gpf.geometric.TerrainFlatteningOp.generateSimulatedImage(TerrainFlatteningOp.java:764)
at org.esa.s1tbx.sar.gpf.geometric.TerrainFlatteningOp.computeTileStack(TerrainFlatteningOp.java:483)
… 48 more
Caused by: java.lang.NullPointerException
at org.esa.s1tbx.sar.gpf.geometric.TerrainFlatteningOp.generateSimulatedImage(TerrainFlatteningOp.java:528)
… 49 moreError: java.lang.NullPointerException