Terrain correction error cannot create layover shadow due to absent of dem

lundj17 · November 8, 2021, 4:31pm

Hi,

I’m running into issues running SNAP via command line using graph batch processing. I’m trying to terrain correct products that are radiometrically terrain flattened, and it looks like there is an issue with the product reader (I’m using only Sentinel-1 data but it keeps on mentioning the RISAT product reader), and mentions the absence of a DEM in creating the layover/shadow mask. I’ll copy and paste below both the xml graph I’m calling with gpt, as well as the errors I’m receiving. Anybody know how to troubleshoot this?
My processing workflow (for Sentinel-1 GRD high res dual pol images) is:
Apply orbit file → Thermal noise removal > GRD border removal > calibrate to beta nought > multi-look > speckle filter > rtf (using Copernicus 30m dem) > tc (using Copernicus 30m dem, saving layover/shadow as well as local incidence angle).
Thanks for any help you can offer!

tc graph xml:

error message (clipped to keep it reasonably short):
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.esa.snap.core.util.EngineVersionCheckActivator: Please check regularly for new updates for the best SNAP experience.
Executing processing graph
SEVERE: org.esa.snap.core.dataio.ProductIO: Error attempting to read S1B_IW_GRDH_1SDV_20210311T035502_20210311T035529_025962_0318C5_3EC5.dim with plugin reader org.esa.s1tbx.io.risat1.Risat1ProductReaderPlugIn@b7d2d51
java.lang.NullPointerException
at org.esa.s1tbx.io.risat1.Risat1ProductReaderPlugIn.findMetadataFile(Risat1ProductReaderPlugIn.java:59)
at org.esa.s1tbx.io.risat1.Risat1ProductReaderPlugIn.findMetadataFile(Risat1ProductReaderPlugIn.java:78)
at org.esa.s1tbx.io.risat1.Risat1ProductReaderPlugIn.getDecodeQualification(Risat1ProductReaderPlugIn.java:44)
at org.esa.snap.core.dataio.ProductIO.getProductReaderForInput(ProductIO.java:304)
at org.esa.snap.core.gpf.common.ReadOp.initialize(ReadOp.java:163)
at org.esa.snap.core.gpf.internal.OperatorContext.initializeOperator(OperatorContext.java:528)
at org.esa.snap.core.gpf.internal.OperatorContext.getTargetProduct(OperatorContext.java:298)
at org.esa.snap.core.gpf.Operator.getTargetProduct(Operator.java:385)
at org.esa.snap.core.gpf.graph.NodeContext.initTargetProduct(NodeContext.java:77)
at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:199)
at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:182)
at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:182)
at org.esa.snap.core.gpf.graph.GraphContext.initOutput(GraphContext.java:166)
at org.esa.snap.core.gpf.graph.GraphContext.(GraphContext.java:85)
at org.esa.snap.core.gpf.graph.GraphContext.(GraphContext.java:58)
at org.esa.snap.core.gpf.graph.GraphProcessor.executeGraph(GraphProcessor.java:118)
at org.esa.snap.core.gpf.main.DefaultCommandLineContext.executeGraph(DefaultCommandLineContext.java:86)
at org.esa.snap.core.gpf.main.CommandLineTool.executeGraph(CommandLineTool.java:547)
at org.esa.snap.core.gpf.main.CommandLineTool.runGraph(CommandLineTool.java:391)
at org.esa.snap.core.gpf.main.CommandLineTool.runGraphOrOperator(CommandLineTool.java:287)
at org.esa.snap.core.gpf.main.CommandLineTool.run(CommandLineTool.java:188)
at org.esa.snap.core.gpf.main.CommandLineTool.run(CommandLineTool.java:121)
at org.esa.snap.core.gpf.main.GPT.run(GPT.java:60)
at org.esa.snap.core.gpf.main.GPT.main(GPT.java:37)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.esa.snap.runtime.Launcher.lambda$run$0(Launcher.java:55)
at org.esa.snap.runtime.Engine.runClientCode(Engine.java:189)
at org.esa.snap.runtime.Launcher.run(Launcher.java:51)
at org.esa.snap.runtime.Launcher.main(Launcher.java:31)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.exe4j.runtime.LauncherEngine.launch(LauncherEngine.java:84)
at com.install4j.runtime.launcher.UnixLauncher.start(UnixLauncher.java:66)
at install4j.org.esa.snap.runtime.Launcher_gpt.main(Unknown Source)

INFO: org.hsqldb.persist.Logger: dataFileCache open start
.Copernicus_DSM_COG_10_N71_00_W159_00_DEM.tif
Copernicus_DSM_COG_10_N71_00_W160_00_DEM.tif
Copernicus_DSM_COG_10_N71_00_W161_00_DEM.tif
Copernicus_DSM_COG_10_N71_00_W162_00_DEM.tif
Copernicus_DSM_COG_10_N71_00_W163_00_DEM.tif
Cannot create layover/shadow mask due to the absent of DEM
Cannot create layover/shadow mask due to the absent of DEM
Cannot create layover/shadow mask due to the absent of DEM
Cannot create layover/shadow mask due to the absent of DEM
Cannot create layover/shadow mask due to the absent of DEM
Cannot create layover/shadow mask due to the absent of DEM
Cannot create layover/shadow mask due to the absent of DEM
Cannot create layover/shadow mask due to the absent of DEM
Cannot create layover/shadow mask due to the absent of DEM
Cannot create layover/shadow mask due to the absent of DEM
Cannot create layover/shadow mask due to the absent of DEM
Cannot create layover/shadow mask due to the absent of DEM
Cannot create layover/shadow mask due to the absent of DEM
Cannot create layover/shadow mask due to the absent of DEM
Cannot create layover/shadow mask due to the absent of DEM
Cannot create layover/shadow mask due to the absent of DEM
Cannot create layover/shadow mask due to the absent of DEM
Cannot create layover/shadow mask due to the absent of DEM
Cannot create layover/shadow mask due to the absent of DEM
Cannot create layover/shadow mask due to the absent of DEM
Cannot create layover/shadow mask due to the absent of DEM
Cannot create layover/shadow mask due to the absent of DEM
Cannot create layover/shadow mask due to the absent of DEM
Cannot create layover/shadow mask due to the absent of DEM
Cannot create layover/shadow mask due to the absent of DEM
Cannot create layover/shadow mask due to the absent of DEM
Cannot create layover/shadow mask due to the absent of DEM
Cannot create layover/shadow mask due to the absent of DEM
Cannot create layover/shadow mask due to the absent of DEM
Cannot create layover/shadow mask due to the absent of DEM
Cannot create layover/shadow mask due to the absent of DEM
Cannot create layover/shadow mask due to the absent of DEM
Cannot create layover/shadow mask due to the absent of DEM
Cannot create layover/shadow mask due to the absent of DEM
Cannot create layover/shadow mask due to the absent of DEM

s0sh0rt · November 22, 2021, 3:37pm

I received the same message: “Cannot create layover/shadow mask due to the absent of DEM” when using an external DEM that did not cover the entire image. Evidently subset in my graph, prior to write, was not limiting the extent of the layover/shadow mask computation which was outside my DEM and subset. The layover/shadow mask produced was all zeros. I did not receive this error using auto-download Copernicus 30m DEMs since all the DEMs required to cover the image were downloaded, and not just for the specified subset.
Your case is a far northern case and may require some special logic for autodownload. Autodownload only covered N71 to N72 and W158 to W163, but your image covers N69.6 to N71.8, and W154.9 to W162.8. The solution for you may be to create a mosaicked superset DEM including N69 to N71 and W155 to W163. Use the new DEM as an external DEM for your processing. Note: External DEMs cannot be compressed, and you will want to apply EGM in Terrain-Correction.

s0sh0rt · November 24, 2021, 7:01pm

Not the best projection for this far north but not having all the DEMs may be causing your errors. The auto-downloaded DEMs are selected (yellow). Another solution is to just download all the DEMs required to cover your image into SNAP’s local cache directory:
“…/auxdata/dem/Copernicus 30m Global DEM/”

fpl · May 23, 2024, 11:41am

Hi there,

we recently found that this issue is still valid in 10.0.0, and I managed to solve it by patching the source code. We checked the resulting mask, and it makes sense: the layover/shadow map is correctly generated.
The error is due to a too much strict check of DEMFactory.getLocalDEM() in sar-op-sar-processing/src/main/java/eu/esa/sar/sar/gpf/geometric/RangeDopplerGeocodingOp.java source. I guess it returns !valid if the DEM contains any tile with nodata pixels (as can happen), and that prevents a correct finalizing of the computation, even if there are tiles with data elsewhere (that’s my guess on the basis of the source code).

Just for the context, this is an issue for Range Doppler Terrain Correction, about the generation of the Layover/Shadow mask.

Hope that other people with the same problem can find this forum entry and find a solution as we did.

diana_harosa · May 30, 2024, 8:30am

@djagula can you have a look at this issue? Thanks!

fpl · June 4, 2024, 7:00am

Just for completion, I simply commented out the !valid check and got the correct result, as expected.
patch

jun_lu · July 11, 2024, 8:55pm

A JIRA ticket ([SNAP-3727] - JIRA) has been created to track the issue. We will look into it. Thank you.

jun_lu · July 12, 2024, 2:09pm

The issue has been fixed and the fix should be available in the next release. Thank you for pointing out the issue.

fpl · December 2, 2024, 10:31am

This is still missing in 11.0.0