S1A product Terrain correction

enemni · January 10, 2021, 7:40pm

Hello,

I have a problem similar to these to S1A product Terrain correction frustration and RuntimeError: java.lang.ArithmeticException: / by zero

Running:

def terrain_correction(product,prj) :
    parameters = HashMap()
    parameters.put('sourceBandNames', 'Amplitude_VV')
    parameters.put('demResamplingMethod', 'CUBIC_CONVOLUTION') 
    parameters.put('imgResamplingMethod', 'CUBIC_CONVOLUTION') 
    parameters.put('demName', 'SRTM 3Sec') 
    parameters.put('pixelSpacingInMeter', 10.0)
    parameters.put('saveSelectedSourceBand', True)
    parameters.put('mapProjection', prj)
    parameters.put('nodataValueAtSea',False)
    parameters.put('maskOutAreaWithoutElevation',False)
    return GPF.createProduct("Terrain-Correction",parameters,product)

It returns

return GPF.createProduct("Terrain-Correction",parameters,product)
RuntimeError: java.lang.ArithmeticException: / by zero

The code below worked using snappy v7. However, we suddenly had the error below as infinite loop and we updated to snap v 8 to solve the below encountering to the error above.

INFO: org.esa.snap.core.dataop.dem.ElevationFile: http retrieving http://cgiar-csi-srtm.openterrain.org.s3.amazonaws.com/source/srtm_44_18.zip
SEVERE: org.esa.snap.core.dataop.dem.ElevationFile: error in opening zip file

Thanks a lot for your help

Best,

enemni

cristianoLopes · January 20, 2021, 9:31am

there have been issues with downloading DEM files and that error message is indicating that.
@ABraun - any ideas por this particular use-case? Python processing.

ABraun · January 20, 2021, 11:34am

have you seen this? SRTM ZIP-files are corrupted or not found

enemni · July 19, 2021, 11:31am

Unfortunately not, I still have these two error:

INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Installed GDAL 3.0.4 set to be used by SNAP.
INFO: org.hsqldb.persist.Logger: dataFileCache open start
WARNING: org.esa.s1tbx.sar.gpf.orbits.ApplyOrbitFileOp: No valid orbit file found for 17-JUL-2021 00:42:43.000000
Orbit files may be downloaded from https://scihub.copernicus.eu/gnss/odata/v1/
and placed in C:\Users\enemni\.snap\auxdata\Orbits\Sentinel-1\POEORB\S1A\2021\07
WARNING: org.esa.s1tbx.sar.gpf.orbits.ApplyOrbitFileOp: Using Sentinel Restituted C:\Users\enemni\.snap\auxdata\Orbits\Sentinel-1\RESORB\S1A\2021\07\S1A_OPER_AUX_RESORB_OPOD_20210717T042731_V20210717T000238_20210717T032008.EOF instead

return GPF.createProduct("Terrain-Correction",parameters,product)
RuntimeError: java.lang.ArithmeticException: / by zero
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: GDAL 3.0.4 found on system. JNI driver will be used.
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Installed GDAL 3.0.4 set to be used by SNAP.
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.esa.snap.core.util.EngineVersionCheckActivator: Please check regularly for new updates for the best SNAP experience.`

Even after upgrading to the latest version of SNAP as suggested in your comment in Gpt sentinel-1 back-geocoding error AND following SRTM ZIP-files are corrupted or not found.

ABraun · July 19, 2021, 7:15pm

this is not surprising, the precise orbit files are only available 2-3 weeks after image acquisition.

enemni · July 20, 2021, 7:19am

Thanks for the prompt reply. We used snappy into our Near-Real-Time (NRT) pipeline and we never had this error or limitation with SNAP v.7. We were always able to process images from the day before. Does this mean that snappy can’t be used for NRT pipeline?

ABraun · July 20, 2021, 7:23am

sorry, this wasn’t clear from your description. Actually, the access to orbit files should work after the update. Maybe you find the flag to disable the raised error if no valid orbit file is found.

enemni · July 20, 2021, 11:14am

Correct, the function

def apply_orbit_file(product):
    parameters = HashMap()
    parameters.put("Orbit State Vectors", "Sentinel Precise (Auto Download)")
    parameters.put("Polynomial Degree", 3)  
    parameters.put("continueOnFail",True)
    return GPF.createProduct("Apply-Orbit-File",parameters,product

works well, the problem is with

def terrain_correction(product,prj) :
    parameters = HashMap()
    parameters.put('sourceBandNames', 'Amplitude_VV')
    parameters.put('demResamplingMethod', 'CUBIC_CONVOLUTION') 
    parameters.put('imgResamplingMethod', 'CUBIC_CONVOLUTION') 
    parameters.put('demName', 'SRTM 3Sec') 
    parameters.put('pixelSpacingInMeter', 10.0)
    parameters.put('externalDEMApplyEGM', True)
    parameters.put('saveSelectedSourceBand', True)
    parameters.put('mapProjection', prj)
    parameters.put('nodataValueAtSea',False)
    parameters.put('maskOutAreaWithoutElevation',False)
    return GPF.createProduct("Terrain-Correction",parameters,product)

that return:

Product name: S1A_IW_GRDH_1SDV_20210717T004347_20210717T004412_038810_049451_96C0
Zip folder already there
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Installed GDAL 3.0.4 set to be used by SNAP.
INFO: org.hsqldb.persist.Logger: dataFileCache open start
WARNING: org.esa.s1tbx.sar.gpf.orbits.ApplyOrbitFileOp: No valid orbit file found for 17-JUL-2021 00:42:43.000000
Orbit files may be downloaded from https://scihub.copernicus.eu/gnss/odata/v1/
and placed in C:\Users\enemni\.snap\auxdata\Orbits\Sentinel-1\POEORB\S1A\2021\07
WARNING: org.esa.s1tbx.sar.gpf.orbits.ApplyOrbitFileOp: Using Sentinel Restituted C:\Users\enemni\.snap\auxdata\Orbits\Sentinel-1\RESORB\S1A\2021\07\S1A_OPER_AUX_RESORB_OPOD_20210717T042731_V20210717T000238_20210717T032008.EOF instead

100% done.
Traceback (most recent call last):
  File "-", line 600, in <module>
    obj.run()
  File "-, line 538, in run
    snap_orb_TC(zip_path,self.prj)
  File "-", line 122, in snap_orb_TC
    VV = terrain_correction(sar_product,prj)
  File "-", line 103, in terrain_correction
    return GPF.createProduct("Terrain-Correction",parameters,product)
RuntimeError: java.lang.ArithmeticException: / by zero
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: GDAL 3.0.4 found on system. JNI driver will be used.
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Installed GDAL 3.0.4 set to be used by SNAP.
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.esa.snap.core.util.EngineVersionCheckActivator: Please check regularly for new updates for the best SNAP experience.

this seems similar to https://forum.step.esa.int/t/orbit-file-timeout-march-2021/28621/153

florian.beyer · July 20, 2021, 1:10pm

As far as I undestood that, yes! you can’t correct your S1 data with precise orbit files in real time (because precise orbit files are available after several weeks)… This means maybe, that your former script (version 7) was looking for precise orbits but never used them caused by the lack of availability. Your script used maybe the restituted orbit files with less accuracy…

Apply the Precise Orbit File
• During the acquisition the satellite position is recorded by a Global Navigation Satellite System (GNSS)
• To assure a fast delivery of Sentinel-1 products orbit information generated by an on-board navigation solution are stored within the Sentinel-1 Level-1 products
• The orbit positions are later refined by the Copernicus Precise Orbit Determination (POD) Service
• Precise orbit files have less than 5 cm accuracy and are delivered within 20 days after data acquisitions
• The accuracy of restituted orbit files is less than 10 cm. The files are in 3 hours after data acquisitions

source:

enemni · July 21, 2021, 7:32am

Unfortunatly, same error with
parameters.put("Orbit State Vectors", "Sentinel Precise (Auto Download)") parameters.put("continueOnFail",True)
or parameters.put("Orbit State Vectors", "Sentinel Restituted (Auto Download)".

florian.beyer · July 21, 2021, 10:57am

You could use the package pyroSAR.
After a lot of problems with snappy, I switched to this package.

The structure and way to write your processing chain is basically the same but much more python style.
pyroSAR Docs
pyroSAR on github

And pyroSAR offers a very stable workaround for the orbit issue…
There is a function implemented. With that you can download the orbit files on the machine where your script is running.
see approach 1 to download orbit files…

You can then use the orbit files on your machine instead trying to download the file during processing.

Once you have your complete preprocessing workflow, you can tweak the parameters and use gpt to execute it… With a bit of luck it is way faster than snappy…
My backscatter (Gamma0) script runs in 5 Minutes instead of hours…

enemni · July 21, 2021, 11:52am

thanks a lot. I actually don’t think my problem is with the orbit file. I think the problem is with the function terrain_correction. To check this, I run Orbit File using SNAP Desktop and I then used the generated file as input for python function terrain_correction(product,'4326'). I got again RuntimeError: java.lang.ArithmeticException: / by zero. I also put a print statement like this:

sar_product = apply_orbit_file(sar_product)
print("Orbit file done")
VV = terrain_correction(sar_product,prj)

and the error come after the print statement showing that the error is with terrain correction function.

enemni · July 21, 2021, 12:40pm

I found aggregation-and-interpolation-of-sentinel-products-should-i-use-snappy-or-gdal-tools which is very similar to my problem .

I commented out this line parameters.put('mapProjection', '4326) from the function terrain_correction(product,prj), which was forcing the projection to be ‘4326’. The solution was inspired by this post “/by zero error” occurs in Terrain-correction(EPSG:4326) and it worked. My python pipeline is now able to retrieve the correct srtm file.

Thanks a lot to the community. I’ve learn a lot about orbit file, thank you @florian.beyer and thanks @ABraun!

ABraun · July 21, 2021, 12:57pm

that’s quite strange, but I’m happy you were able to solve it.
@jun_lu Does this make sense to you, and can this be avoided somehow?