Snappy error - ProductIO.writeProduct

Hi,

I’m trying to implement a simple split of a Sentinel-1 product using snappy.
Here you can see my script:

import sys

sys.path.append('C:\\Users\\juanesburgo\\Desktop\\temp\\snap_python')

import snappy
from snappy import ProductIO
from snappy import jpy
from snappy import GPF

ProductIOPlugInManager = snappy.jpy.get_type('org.esa.snap.core.dataio.ProductIOPlugInManager')
Logger = jpy.get_type('java.util.logging.Logger')
Level = jpy.get_type('java.util.logging.Level')
Arrays = jpy.get_type('java.util.Arrays')
File = jpy.get_type('java.io.File')
String = jpy.get_type('java.lang.String')
HashMap = jpy.get_type('java.util.HashMap')

Logger.getLogger('').setLevel(Level.OFF)
snappy.SystemUtils.LOG.setLevel(Level.OFF)

reader_spi_it = ProductIOPlugInManager.getInstance().getAllReaderPlugIns()
writer_spi_it = ProductIOPlugInManager.getInstance().getAllWriterPlugIns()
GPF.getDefaultInstance().getOperatorSpiRegistry().loadOperatorSpis()

# file paths
ifile = 'C:\\Users\\juanesburgo\\Desktop\\temp\\dados\\S1A_IW_SLC__1SDV_20150903T050105_20150903T050132_007546_00A6E6_526A\\manifest.safe'
ofile = 'C:\\Users\\juanesburgo\\Desktop\\temp\\dados\\res\\snappy_sliped_output.dim'

# read
source = ProductIO.readProduct(ifile)

name = source.getName()
w = source.getSceneRasterWidth()
h = source.getSceneRasterHeight()
print( w, "x", h, "pixels")

# split
parameters = HashMap()
parameters.put('subswath', 'IW1')
parameters.put('selectedPolarisations', 'VV')
sliped_product = GPF.createProduct('TOPSAR-Split', parameters, source)

w = sliped_product.getSceneRasterWidth()
h = sliped_product.getSceneRasterHeight()
print( w, "x", h, "pixels")

# write
ProductIO.writeProduct(sliped_product, ofile, "BEAM-DIMAP")

When I run it I get this output:

74696 x 13716 pixels
22719 x 13563 pixels
Traceback (most recent call last):
  File "C:/Users/juanesburgo/Desktop/temp/snap_python/snappy/examples/MeusScripts/tentativa4.py", line 53, in <module>
    ProductIO.writeProduct(sliped_product, ofile, "BEAM-DIMAP")
RuntimeError: java.lang.OutOfMemoryError: Java heap space

As you can see the split is done but I can’t write the output.

Any sugestion?

Thanks.

I’m not sure, I’m not very familiar with snappy, but I think you have to set a value in the jpyconfig.py script in the snappy folder.
Try setting
jvm_maxmem = None
to
jvm_maxmem = '6G'

Another place to change the memory usage might be in snappy.ini in the snappy folder.
Change the line
# java_max_mem: 4G
to e.g.
java_max_mem: 6G

5 Likes

Marpet, thank you.
:smile:

I am still unable to tackle with the error or memory exceeded. is there any suggestion please?

Thanks Marpet…i got it i was on mistake :slight_smile:

Hello @marpet hope you are doing well,
I have one challenge in sentinel1 preprocessing using snappy python,
I am used one tool “Apply-Orbit-File” using manual process with SNAP Desktop as well as with the help of snappy python everything is working fine,

when I am trying to compare both output (manual and python) in Qgis software the pixel value does not match at all. I just need the output which is match with snappy python output.

can you please let me know where i have to focus on this,

The source code I have given below for your reference ;

def ApplyOrbitFuncton(S1_read, out_path):
    params = snappy.HashMap()
    params.put('orbitType',"Sentinel Precise (Auto Download)")
    params.put('polyDegree',3)
    params.put('continueOnFail', False)
    apply_orbit = snappy.GPF.createProduct('Apply-Orbit-File',params,subset)
    
    nme = "Orbit_1259"
    out = os.path.join(out_path,nme)

    snappy.ProductIO.writeProduct(apply_orbit,out, 'GeoTIFF-BigTIFF')
    print("2 Apply orbit succesfully done")
    return apply_orbit

I used all the parameters as it is used in manual process in SNAP

You are saying the values do not match at all.
Have you also stored the data to GeoTIFF-BigTIFF when using the Desktop and have you reload it?
It could be that the scaling factor of the bands, if they are scaled, makes the difference.
In GeoTiff only raw values are stored. Maybe you can also work with NetCDF4-CF or store the data to BEAM-DIMAP?

1 Like

Sir it was fix that i have to create output same as SNAP Desktop makes,

I need to make changes in code only not in manual process

my main aim is to make output same as manual process

I have changed the output from GeoTIFF-BigTIFF to GeoTiff

but there are no changes detected.

@marpet @juanesburgo
Hello everyone, Hope you all are doing well,

please let me suggest the solution for my problem that I have mention.
I have made one mistake at the first post at that time I said “I just need the output which is match with snappy python output.” but it was my mistake ,

actually I need the output which will be the same as per SNAP Desktop output.

@lveci Do you have an idea why the results are different in both scenarios?

Problem resolved in 4steps, but same problem occur in 5th step in terrain correction;

  1. apply orbit
  2. thermal noise
  3. calibration
  4. speckle filter
  5. terrain correction
  6. linear to formdb
  7. write
def terrain_correction(Speckle,out_path):
    params = snappy.HashMap()
    params.put('sourceBands','Sigma0_VH,Sigma0_VV')
    params.put('demName', 'SRTM 3Sec')
    params.put('demResamplingMethod','BILINEAR_INTERPOLATION')
    params.put('imgResamplingMethod','BILINEAR_INTERPOLATION')
    params.put('pixelSpacingInMeter', 10.0)
    params.put('pixelSpacingInDegree',0.00008983152841195215)
    params.put('mapProjection','WGS84(DD)')
    params.put('saveSelectedSourceBand',True)
    
    terrain = snappy.GPF.createProduct('Terrain-Correction', params, Speckle)
    
    nme = "Orbit_Thermal_Cali_Speckle_Terrain_test1"
    out = os.path.join(out_path,nme)
    Config = snappy.jpy.get_type('org.esa.snap.runtime.Config')
    Config.instance().preferences().put('snap.dataio.bigtiff.compression.type', 'LZW')
#    Config.instance().preferences().put('snap.dataio.bigtiff.compression.quality', '0.75')
    snappy.ProductIO.writeProduct(terrain,out, 'GeoTIFF-BigTIFF')
    print("6 terrain succesfully done")    
    return terrain

there is minor changes detected in pixel values of manual output and snappy output
@marpet @lveci can you please let me know if you have any idea ?