Using mask file with KMeansClusterAnalysis in python


#1

How do I assign a mask from a shapefile when using KMeansClusterAnalysis in python:

I have the following so far, but I get the error below:

source = ProductIO.readProduct(mask_path)
parameters = HashMap()

parameters.put('vectorFile', river_mask_file)
target = GPF.createProduct('Import-Vector', parameters, source)

source = ProductIO.readProduct(in_file)
parameters.put('clusterCount', 30)
parameters.put('iterationCount', 30)
parameters.put('randomSeed', 314150)
parameters.put('roiMaskName', 'MASK_River_dissolve')
parameters.put('sourceBandNames', in_band)
target = GPF.createProduct('KMeansClusterAnalysis', parameters, source)
ProductIO.writeProduct(target, out_file, 'GeoTIFF')

Error:
target = GPF.createProduct(‘Import-Vector’, parameters, source)
RuntimeError: ambiguous Java method call, too many matching method overloads found

It works in the GUI version by dragging the shapefile in, but I’d like it to work in python.
Any help would be much appreciated.
Thanks
John


Subset of Sentinel-1A images
#2

Where does mask_path point to? Maybe it is not a product which can be read by SNAP.
Is source properly initialised after doing this call? I think it could be None.


#3

If the ‘source’ is initialised with the geotif file that I want masked… see below, I get the following error:
target = GPF.createProduct(‘Import-Vector’, parameters, source)
RuntimeError: org.esa.snap.core.gpf.OperatorException: com/bc/ceres/swing/figure
/FigureStyle

I have tried initialised ‘source’ with the shapefile but I get the following error:
target = GPF.createProduct(‘Import-Vector’, parameters, source)
RuntimeError: ambiguous Java method call, too many matching method overloads found

parameters = HashMap()
in_file = r"D:\John\projects\river\input.tif"
river_mask_file = r"D:\John\projects\river\input.tif"
source = ProductIO.readProduct(in_file)
parameters.put('vectorFile', river_mask_file)
target = GPF.createProduct('Import-Vector', parameters, source)
source = ProductIO.readProduct(in_file)
parameters.put('clusterCount', 30)
parameters.put('iterationCount', 30)
parameters.put('randomSeed', 314150)
parameters.put('roiMaskName', 'MASK_SlaveRiver_dissolve')
parameters.put('sourceBandNames', in_band)
target = GPF.createProduct('KMeansClusterAnalysis', parameters, source)
ProductIO.writeProduct(target, out_file, 'GeoTIFF')

#4

river_mask_file needs to be a shapefile (*.shp).
The in_file should be OK as it is.
For the call of KMeansClusterAnalysis you should exchange source by target. sourceis the original unchanged file.


#5

Hi Marc
Sorry I had a typo in my question … the river mask was defined as a shapefile:
river_mask_file = r"D:\John\projects\river\input.shp"
and
in_file = r"D:\John\projects\river\input.tif"
source = ProductIO.readProduct(in_file)

and I get the following error:

target = GPF.createProduct(‘Import-Vector’, parameters, source)
RuntimeError: org.esa.snap.core.gpf.OperatorException: com/bc/ceres/swing/figure
/FigureStyle


#6

I think this is caused by an error in the snappy initialisation.

Please have a look in the __init__.py file of snappy.

You will see the following code at line 25-26

EXCLUDED_DIR_NAMES = {'org.esa.snap.snap-worldwind', 'org.esa.snap.snap-rcp', 'org.esa.snap.snap-product-library',
                  'org.esa.snap.ceres-ui', 'org.esa.snap.snap-sta-ui'}

Please remove the ‘org.esa.snap.ceres-ui’ from the list.

Afterwards it should work


#7

Hi Marco
I removed ‘org.esa.snap.ceres-ui’ from line 25-26 of init.py, but I’m getting the same error:
target = GPF.createProduct(‘Import-Vector’, parameters, source)
RuntimeError: org.esa.snap.core.gpf.OperatorException: com/bc/ceres/swing/figure
/FigureStyle
Any other suggestions?
Thanks
John


Error in SNAP
#8

Probably this wasn’t enough, the needed jar file is still not loaded.
In the same file try to change following lines (90-93)

...
if os.path.isfile(path) and name.endswith('.jar'):
if not (name.endswith('-ui.jar') or name in EXCLUDED_JAR_NAMES):
    env[0].append(path)
elif os.path.isdir(path) and name not in EXCLUDED_DIR_NAMES:
...

to

...
if os.path.isfile(path) and name.endswith('.jar'):
    env[0].append(path)
elif os.path.isdir(path):
...

#9

Thanks Marco
After making the changes that you suggested to init.py the following python worked successfully :

parameters = HashMap()
source = ProductIO.readProduct(in_file)
parameters.put('vectorFile', river_mask_file)
parameters.put('separateShapes', False)
target1 = GPF.createProduct('Import-Vector', parameters, source)

parameters.put('clusterCount', 30)
parameters.put('iterationCount', 30)
parameters.put('randomSeed', 314150)
parameters.put('roiMaskName', 'MASK_SlaveRiver_dissolve')
parameters.put('sourceBandNames', in_band)
target2 = GPF.createProduct('KMeansClusterAnalysis', parameters, target1)
ProductIO.writeProduct(target2, out_file, 'GeoTIFF')

I did get the following warning after making your suggested changes to init.py (any suggestions to get rid of these warnings?)

WARNING: org.esa.snap.SnapCoreActivator: org.esa.snap.core.gpf.OperatorSpi: Prov
ider org.esa.snap.python.gpf.PyOperatorSpi could not be instantiated
java.lang.NoClassDefFoundError: org/openide/util/Lookup
at org.esa.snap.rcp.util.NbResourceLocator.getResourceClassLoader(NbReso
urceLocator.java:22)
at com.bc.ceres.core.DefaultResourceLocator.locateResources(DefaultResou
rceLocator.java:31)
at com.bc.ceres.core.ResourceLocator.getResources(ResourceLocator.java:3
4)
at org.esa.snap.core.util.ServiceFinder.scanClassPath(ServiceFinder.java
:195)
at org.esa.snap.core.util.ServiceFinder.findServices(ServiceFinder.java:
148)
at org.esa.snap.python.gpf.PyOperatorSpi.(PyOperatorSpi.java:42)

    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

    at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)

    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Sou

rce)
at java.lang.reflect.Constructor.newInstance(Unknown Source)
at java.lang.Class.newInstance(Unknown Source)
at java.util.ServiceLoader$LazyIterator.nextService(Unknown Source)
at java.util.ServiceLoader$LazyIterator.next(Unknown Source)
at java.util.ServiceLoader$1.next(Unknown Source)
at org.esa.snap.SnapCoreActivator.loadServices(SnapCoreActivator.java:60
)
at org.esa.snap.core.gpf.internal.OperatorSpiRegistryImpl.loadOperatorSp
is(OperatorSpiRegistryImpl.java:77)
Caused by: java.lang.ClassNotFoundException: org.openide.util.Lookup
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
… 16 more

Start processing…


#10

Hi Marco
Is it possible to obtain any statistical info (mean, SD, etc… for each input layer) under each class generated by KMeansClusterAnalysis?
Let me know
Thanks for all your help
John


#11

OK, good that it is running. The error might occurs because we now include to many modules. So we must be more specific again.

...
if os.path.isfile(path) and name.endswith('.jar'):
if name.endswith('ceres-ui.jar') or (not (name.endswith('-ui.jar') orname in EXCLUDED_JAR_NAMES)):
    env[0].append(path)
elif os.path.isdir(path) and name not in EXCLUDED_DIR_NAMES:
...

Probably you also have to remove ‘org.esa.snap.ceres-ui’ from the EXCLUDED_DIR_NAMES again.

In the metadata you find a section named Cluster_Analysis
There you find for each class the cluster centers of each of the input bands.
But more information is not provided.


#12

Awesome, the code changes worked, no more errors

Where is the metadata located? It only creates a geotiff, if I run gdalinfo on the geotiff I get the following metadata info:

Metadata domains:
(default)
IMAGE_STRUCTURE
Metadata:
AREA_OR_POINT=Area
TIFFTAG_IMAGEDESCRIPTION=FQ17W_20151214_135725_class_snappy
TIFFTAG_RESOLUTIONUNIT=1 (unitless)
TIFFTAG_XRESOLUTION=1
TIFFTAG_YRESOLUTION=1


#13

If you open the file with SNAP you should see the metadata.
For external applications it might be a bit complicated to access this metadata.
SNAP writes some XML into a Tiff_Tag field with the number 65000 (if I remember correctly). Inside this the Cluster_Analysis metadata is stored.


#14

Hello,

I had the same issues as above, and followed the suggested changes to init.py:

  1. Remove ‘org.esa.snap.ceres-ui’ from EXCLUDED_DIR_NAMES

  2. Change
    if os.path.isfile(path) and name.endswith(’.jar’):
    if not (name.endswith(’-ui.jar’) or name in EXCLUDED_JAR_NAMES):
    env[0][name] = path
    elif os.path.isdir(path) and name not in EXCLUDED_DIR_NAMES:
    to
    if os.path.isfile(path) and name.endswith(’.jar’):
    env[0][name] = path
    elif os.path.isdir(path):

I have the latest version of SNAP installed.
When I try to import snappy functionality I get the following:

>>> from snappy import ProductIO, HashMap, GPF

WARNING: org.esa.snap.core.util.ServiceLoader: org.esa.snap.core.gpf.OperatorSpi: Provider org.esa.snap.python.gpf.PyOperatorSpi could not be instantiated

java.lang.NoClassDefFoundError: org/openide/util/Lookup

at org.esa.snap.rcp.util.NbResourceLocator.getResourceClassLoader(NbResourceLocator.java:22)

at com.bc.ceres.core.DefaultResourceLocator.locateResources(DefaultResourceLocator.java:31)

at com.bc.ceres.core.ResourceLocator.getResources(ResourceLocator.java:34)

at org.esa.snap.core.util.ServiceFinder.scanClassPath(ServiceFinder.java:195)

at org.esa.snap.core.util.ServiceFinder.findServices(ServiceFinder.java:148)

at org.esa.snap.python.gpf.PyOperatorSpi.<clinit>(PyOperatorSpi.java:44)

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)

at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

at java.lang.reflect.Constructor.newInstance(Constructor.java:423)

at java.lang.Class.newInstance(Class.java:442)

at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)

at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)

at java.util.ServiceLoader$1.next(ServiceLoader.java:480)

at org.esa.snap.core.util.ServiceLoader.loadServices(ServiceLoader.java:40)

at org.esa.snap.core.gpf.internal.OperatorSpiRegistryImpl.loadOperatorSpis(OperatorSpiRegistryImpl.java:77)

at org.esa.snap.core.gpf.GPF.<clinit>(GPF.java:97)

Caused by: java.lang.ClassNotFoundException: org.openide.util.Lookup

at java.net.URLClassLoader.findClass(URLClassLoader.java:381)

at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)

at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

… 17 more

Traceback (most recent call last):

File “<stdin>”, line 1, in <module>

File “/Users/kamen/anaconda2/lib/python2.7/site-packages/snappy/init.py”, line 395, in <module>

Engine.start()

RuntimeError: java.util.ServiceConfigurationError: org.esa.snap.runtime.Activator: Provider org.esa.snap.worldwind.WWActivator could not be instantiated


#15

Also, I’d like to point out that another issue may be that the function call
sentinel_1 = ProductIO.readProduct(output_dir + ‘manifest.xml’)
returns a NoneType object…