What is the recommended solution for the issue of black blocks in the middle of terrain-flattened Sentinel-1 images? Excessive multilooking is not a solution. I have tested 5m resolution DEMs with ‘non-regrid’ which fails with null pointer exception. There is no change with version 6.0. I am working with S1A_IW_GRDH_1SSV_20160526T194932_20160526T195003_011434_011670_F15D
Additional information: The minimal process is read-calibration-terrainflatten-write. Using various downloaded, or my DEM achieves a similar result. DEMs do not have zeros or fill values. 10 of the 47 S1A, relative orbit 162, images over the mountains near Port Moresby, have large black blocks in the same or similar areas of the terrain-flattened images. Most of the others have no defects, except a few that contain much smaller defects near shadow areas. Below is a small subset of the 20160526 flattened image.
some black areas are introduced when 0 is set as a NoData value in the Raster properties. If these zeroes are processed they are treated as NoData and sometimes get larger (when resampling is involved).
There are no NoData areas in our DEMs.
Don’t use the original method. It should only be ‘re-grid’. When processing a tile, snap retrieves the dem over the area that should cover the tile. With SAR and mountains, considerable overlap may be needed. The artifacts look like there wasn’t enough dem for that tile. We would need to increase the amount of dem overlap or make it a parameter.
I just tested the 6.0 release and found similar black holes, compared to 5.0 and 6.0 Beta, in the terrain flattened product for: S1A_IW_GRDH_1SSV_20160526T194932_20160526T195003_011434_011670_F15D
Luis suggested that the in memory DEM tiles might need to be increased in size. I think this could alternately be the result of not enough, or not the correct set, of in memory S1 tiles being loaded for some areas. Notice the left edge of these black areas follows the topography similar to the edge of the ortho images.
Another terrain-flattening black area effect, “edge-wedges”, that appears on the left and right side of some terrain flattened products is shown last below.
Mapping of some areas of New Guinea cannot be completed without a fix for this issue.
Left side of terrain flattened image:
I am using SNAP 6.0 for processing S1A GRDH data and after terrain flattening my results are affected by artifacts too. Before doing the flattening there are just low intensity values located. After the flattening those areas are 0.
The datasets I used are:
Here are the processing steps:
- Remove-GRD-Border-Noise (I tried it without this step, but the results were the same)
- Calibration (beta naught)
- Terrain Flattening (SRTM 1 Sec HGT Auto Download; bicubic; Re-grid)
Figure 1: SRTM 1 sec
Figure 2: Dataset 20160302 after processing steps 1 to 5
Figure 3: Dataset 20160829 after processing steps 1 to 5
Figure 1: Dataset 20160302 after processing steps 1 to 4
Figure 2: Dataset 20160302 after processing steps 1 to 5 (SRTM 1sec)
Figure 3: Dataset 20160302 after processing steps 1 to 5 (SRTM 3sec): less artifacts than by using the 1sec SRTM
Thereafter I want to apply the following steps:
6. Range-Doppler Terrain Correction (SRTM 1 Sec HGT Auto Download; DEM: Bilinear Resampling; Image Resampling: Nearest Neighbour; 10m)
7. Create Stack (Resampling: None; Initial Offset: Orbit) of 10 datasets (8 x descending orbit 139; 2 x ascending orbit 88)
9. Multitemporal Speckle Filtering (NEST; 7x7)
10. Linear to dB
Do you have any ideas for solving this problem? I tested many combinations of the processing steps, but all results are affected. Do you need more information? After the preprocessing I am going to do a classification of tree species. which is why I want to reduce radiometric distortions. As I am using datasets of different orbits, terrain flattening should be an essential step.
Have I overlooked something or does nobody have a solution?
Based on the idea by Small (2011), am I right that TF should be applicable to each pixel in the dataset without having artifacts?
I did another test for finding out what’s the problem with TF.
I used SRTM 1Sec HGT (Auto Download) for downloading all relevant tiles, switched off the internet and renamed the files. Just one tile retained its name (N50E007). The location of the tiles is:
"C:\Users\username\.snap\auxdata\dem\SRTM 1Sec HGT". Despite editing the folder paths (
"C:\Program Files\snap\etc\snap.auxdata.properties"), an external DEM or the option SRTM 1Sec Grid are not working.
The figure in the lower left shows the result of TF using the replaced SRTM tiles. For N50E007 the TF was applied correctly (but artifacts are visible). For all other tiles the replaced ones are used, because new terrain effects occur after TF, but the black area artifacts are gone.
In the following figures the subset of my previous post is shown.
Figure 1: 1. Remove-GRD-Border-Noise -> 2. ThermalNoiseRemoval -> 3. Apply-Orbit-File -> 4. Calibration (beta naught)
Figure 2: 1. Remove-GRD-Border-Noise -> 2. ThermalNoiseRemoval -> 3. Apply-Orbit-File -> 4. Calibration (beta naught) -> TF (SRTM 1Sec; correct tiles) || Artifacts are visible
Figure 3: 1. Remove-GRD-Border-Noise -> 2. ThermalNoiseRemoval -> 3. Apply-Orbit-File -> 4. Calibration (beta naught) -> TF (SRTM 1Sec; replaced tiles) || Artifacts are NOT visible
For figure 3 the SRTM tile N49E006 was replaced by N49E007, whereby no artifacts occurred.
Same problem with TF here. Bad over mountainous areas.
TOPSAR-split-aply orbit-thermal noise-calibration to beta-terrain flatten (No regrid, bicubic interpolation, SRTM1 auto download.)
Regrid UNFLAGGED because with grid it simply makes processing takes too long (2min -> 4 hours)
Oh man, the artifacts are shadow areas. Sorry!
Shadow areas would not have straight line edges. I am guessing that those straight edges are the edges of missing DEM tiles in memory.
I agree with you that there are probably DEM tiles missing. My post referred to the problem I had, which were shadows at the end.
unfortunately also the re-grid method shows this error. I processed two consecutive SLC scenes ( since I am interestd also in Coherence in my workflow) over Andes in Ecuador with the following workflow:
SliceAssembly --> Split to subswaths --> Apply Orbit File --> Remove TN --> Calibrate beta --> Deburst --> TF
btw…the re-grid method takes 5 hours per subswath, i…e 15 hours for the 2 consecutive scenes on a 8 core, 32gb ram machine…is this normal? any idea on how to speed this up?
You were mentioning that the DEM overlap can be easily increased, or implemented as a parameter. Is this planned soonish?
Thanks in advance!
are there any news on this topic? I think this is quite a critical bug, since it hinders the production of ARD data for land cover classification.
Regridding method does sound anomalously slow @lveci . We’re planning to revamp Terrain Flattening but it’s not clear yet when that will take place.
In addition, regridding did not solve the issue of black areas.
I see my SNAP GUI v6.0.7 does not have the regrid toggle anymore, and when looking at the graph the regrid method is set to true.
Is the regrid = true method for Terrain Flattening now the recommended default method?
I ask because I had a snappy terrain correction program working using regrid=false on SNAP v6.0.0. I had to update SNAP to fix SliceAssembly and this seems to have broken my terrain correction workflow (java null pointer exception) and I suspect it is due to the regrid method that should now be true.
Thank you for adding the new overlap percentage parameters.
EDIT: Changing regrid to True was required for TF after I updated snap from v6.0.0. and TF works now
Geocell S11W077 is a classic example over the Andes processed with S1TBX 6.0.4:.
I updated SNAP on my linux server. S1TBX version is now 6.0.6, adding both new Terrain-Flattening parameters to my gpt xml:
Images used:dditionalOverlap = 1
oversamplingMultiple = 1.5
I now receive NullPointerException regardless of the setting of reGridMethod.
DEM= SRTM geocell s11w077
Version 7.0 does not exhibit the problem shown above. However the Terrain-Flattening default additionalOverlap=0.1 left black ~triangular regions on the left edge of the processed area, and the right edge of the S1 image. The black regions diminished at 0.3 and 0.5, and were gone at additionalOverlap=0.8. This is possibly because of the large elevation change (1378m to 6470m) in this area of the Andes near Yerupajá 10°16′S 76°54′W.
additionalOverlap=0.1, 0.3, and 0.8 shown below: