Error with Subset using geoRegion

huw · March 5, 2021, 11:06am

Hello step-forum community,
I want to subset a region from a Sentinel-1 product using longitude and latitude coordinates. Here’s what I have so far:

import snappy
from snappy import WKTReader

product_path = '/home/test/Documents/insar/iceland/original/'

input_S1_files = sorted(list(iglob(join(product_path, '**', '*S1*.zip'),
                                   recursive=True)))

name, sensing_mode, product_type, polarization, height, width, band_names = (
    [] for i in range(7))

for i in input_S1_files:
    sensing_mode.append(i.split("_")[3])
    product_type.append(i.split("_")[4])
    polarization.append(i.split("_")[-6])
    s1_read = snappy.ProductIO.readProduct(i)
    name.append(s1_read.getName())
    height.append(s1_read.getSceneRasterHeight())
    width.append(s1_read.getSceneRasterWidth())
    band_names.append(s1_read.getBandNames())

wkt = "POLYGON((-21.962 64.708,-21.962 64.871,-22.248 64.871,-22.248 54.708,-21.962 64.708))"
geom = WKTReader().read(wkt)
parameters = snappy.HashMap()
parameters.put('copyMetadata', True)
parameters.put('geoRegion', geom)
subset = snappy.GPF.createProduct('Subset', parameters, s1_read)

However, I get the following RuntimeError:

File " ", line , in module 
subset = snappy.GPF.createProduct('Subset', parameters, s1_read)

RuntimeError: org.esa.snap.core.gpf.OperatorException: The pixel region 'java.awt.Rectangle[x=0,y=0,width=0,height=0]' is invalid.

What do you think is happening and what can I do to solve the error?

Thanks

gnwiii · March 5, 2021, 3:50pm

The example script (~/.snap/snap-python/snappy/Examples/snappy_subset.py) has a different way to set parameters for the Subset operator.

marpet · March 7, 2021, 8:21am

Difficult to say what the problem is.
Either the region does not intersect with one of your products, but then the error message should actually be different or the source product is not healthy and its dimensions are zero.

Does this happen with every product or just with one of them?

huw · March 8, 2021, 4:22am

Hello @gnwiii,
Thanks for responding. I ran the script snappy_subset.py:

import sys

import snappy
from snappy import ProductIO, WKTReader

SubsetOp = snappy.jpy.get_type('org.esa.snap.core.gpf.common.SubsetOp')

if len(sys.argv) != 3:
    print("usage: %s <file> <geometry-wkt>" % sys.argv[0])
    print("       %s ./TEST.N1 \"POLYGON((15.786082 45.30223, 11.798364 46.118263, 10.878688 43.61961, 14.722727"
          "42.85818, 15.786082 45.30223))\"" % sys.argv[0])
    sys.exit(1)

file = sys.argv[1]
wkt = sys.argv[2]

geom = WKTReader().read(wkt)

print("Reading...")
product = ProductIO.readProduct(file)

op = SubsetOp()
op.setSourceProduct(product)
op.setGeoRegion(geom)
op.setCopyMetadata(True)

sub_product = op.getTargetProduct()

print("Writing...")
ProductIO.writeProduct(sub_product, "snappy_subset_output.dim", "BEAM-DIMAP")

print("Done.")

I get the following error:

INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: GDAL 3.0.4 found on system. JNI driver will be used.
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Installed GDAL 3.0.4 set to be used by SNAP.
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
usage:  <file> <geometry-wkt>
        ./TEST.N1 "POLYGON((15.786082 45.30223, 11.798364 46.118263, 10.878688 43.61961, 14.72272742.85818, 15.786082 45.30223))"
An exception has occurred, use %tb to see the full traceback.

SystemExit: 1


INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: GDAL 3.0.4 found on system. JNI driver will be used.
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Installed GDAL 3.0.4 set to be used by SNAP.
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.esa.snap.core.util.EngineVersionCheckActivator: Please check regularly for new updates for the best SNAP experience.

Any idea what’s happening?

Thanks

huw · March 8, 2021, 4:24am

Hello @marpet,

Thanks for your prompt response. You’re right, I wrote the incorrect WKT string. The number 54.708 in the WKT string should be 64.708. This solves the original error.

However, I now get a new error:

INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: GDAL 3.0.4 found on system. JNI driver will be used.
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Installed GDAL 3.0.4 set to be used by SNAP.
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.esa.snap.core.util.EngineVersionCheckActivator: Please check regularly for new updates for the best SNAP experience.
                                                Name Sensing Mode  \
0  S1A_IW_SLC__1SDV_20180623T185856_20180623T1859...                
1  S1A_IW_SLC__1SDV_20180717T185857_20180717T1859...                

  Product Type Polarization  Height  Width  \
0         1SDV         1SDV   13662  69691   
1         1SDV         1SDV   13662  69691   

                                          Band Names  
0  [i_IW1_VH, q_IW1_VH, Intensity_IW1_VH, i_IW1_V...  
1  [i_IW1_VH, q_IW1_VH, Intensity_IW1_VH, i_IW1_V...  
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Installed GDAL 3.0.4 set to be used by SNAP.
Traceback (most recent call last):

  File "/home/huw/Dropbox/Scripts/insar_iceland.py", line 183, in <module>
    subset = snappy.GPF.createProduct('Subset', parameters, s1_read)

RuntimeError: org.esa.snap.core.gpf.OperatorException

What do you think is happening?

Thanks

marpet · March 8, 2021, 7:28am

Can you surround your code with exception handling?

Like

try:
  subset = snappy.GPF.createProduct('Subset', parameters, s1_read)
except OperatorException as e:
    print(e.getMessage())

This will probably tell more about the reason.

huw · March 9, 2021, 3:40am

@marpet
Is there a module that I need to import which defines ‘OperatorException’?

NameError: name 'OperatorException' is not defined

marpet · March 9, 2021, 7:36am

Oh yes, I forgot.

You need to load the type once:

OperatorException= jpy.get_type('org.esa.snap.core.gpf.OperatorException')

huw · March 9, 2021, 11:08am

@marpet

After running this:

OperatorException= jpy.get_type('org.esa.snap.core.gpf.OperatorException')

try:
  subset = snappy.GPF.createProduct('Subset', parameters, s1_read)
except OperatorException as e:
    print(e.getMessage())

The following error is returned:

TypeError: catching classes that do not inherit from BaseException is not allowed

marpet · March 9, 2021, 12:16pm

Oh, sorry this happens when making suggesting without testing them beforehand.

Sure, a Java Exception (OperatorException) can’t be caught by Python.
So, you need to catch RuntimeException for example.

I’ve once suggested similar here: How do you handle exceptions and warnings in snappy? - snap - STEP Forum (esa.int)

But I’m not sure at the moment if you can really get to the message of the Java exception.
Maybe you can investigate the exception you catch, which type is it and what attributes does it provide.

It could be that exceptions need to be handled differently in SNAPPY/JPY to provide more inforamtion.

If the above does not help I can only suggest to try your configuration on the command line with gpt and see if the error also occurs.

huw · March 10, 2021, 5:54am

I tried your suggestion in the link provided

try:
  subset = snappy.GPF.createProduct('Subset', parameters, s1_read)
except RuntimeError as e:
    print(e)

returns

org.esa.snap.core.gpf.OperatorException

You’re right there’s not much I can do with this.

Regarding your second suggestion: configuration on the command line with gpt. Unfortunately, that goes beyond the scope of my programming knowledge. Do you know of tutorials where I can learn to use gpt on the command line?

marpet · March 10, 2021, 6:52am

Actually, it is simpler than using python.
You find tutorials here:
Bulk Processing with GPT - SNAP
Creating a GPF Graph - SNAP

Instead of creating the graph manually as described in the tutorial above, you can create template graph with the Graph Builder which can be adapted.
If you like you can use it directly in the GUI with the Batch Processing tool.

gnwiii · March 10, 2021, 12:47pm

It would be helpful to have some guidance on when python and SNAP snappy should be preferred over GPT. Certainly snappy is useful when you want to do something that isn’t available from GPT using numpy, pandas, etc. Python complicates error handling and memory management. When running the same GPF processing steps on a list of input files, using GPT in a shell script that loops over the list of input files is generally better than using Python.

I started using computers when programs were entered using punched cards or paper tape, so access to a command-line was a major advance. Today, many users have very little command-line experience but have learned programming with python in an IDE. It is very useful to spend some time learning to use a comand-line environment. PowerShell on Windows, POSIX shells (bash, zsh, dash) on linux and macOS (or Windows via cygwin, Msys2, or WSL) are very useful for troubleshooting and batch processing.