Example script for multiple operations?


#1

I was wondering if anyone has an example of how to write a python snappy script with multiple sequential operators? There’s also a strong chance I’m misunderstanding exactly how to work with snappy. I’ve tried examining a few examples such as NDVI_OP (which I don’t understand how to use) and snappy_flh (which I think I understand but am not sure what multispectral sensor input is required).

I’ve also looked at @abgbaumann’s snappy_glcm , and it seems like I could repeat/modify that (func_name & func_code) to cover each operator I need, then have another script that calls each operator in a sequence on the file. Is that the best approach?

What I’d like to make in the end is a script that will:

INGEST sentinel-1 scene
CALIBRATE sentinel-1 scene
TERRAIN FLATTEN calibrated sentinel-1 scene
SPECKLE FILTER terrain-cal sentinel 1 scene
GEORECTIFY speckle-terrain-cal sentinel 1 scene
OUTPUT geo-speckle-terrain-cal sentinel 1 scene

Thank you!


Operate GPT in python
GPT Sticky Needed in the SNAP Forum
Sen2cor and python module
Oil spill detection - using SNAPpy
Compute zonal statistics from Sentinel-2 VI's
Reprojection s3 slstr issue
#2

You can easily concatenate the operators.

You invoke each operator by calling GPF.createProduct('OperatorName', parameter, source_product). This will give you a new product which you can use as source product for the next operator.

This is some sample code:

HashMap = snappy.jpy.get_type('java.util.HashMap')    
snappy.GPF.getDefaultInstance().getOperatorSpiRegistry().loadOperatorSpis()
source = snappy.ProductIO.readProduct('G:/EOData/Meris/RR/MER_RR__1PNBCG20050709_101121_000001802038_00466_17554_0001.N1')

parameters = HashMap()
parameters.put('computeCHL', False)
result1 = snappy.GPF.createProduct('FubWeWOp', parameters, source)
parameters = HashMap()
parameters.put('otherParam', 3.0)
result2 = snappy.GPF.createProduct('Op2', parameters, result1 )
.... # more operators here
snappy.ProductIO.writeProduct(result2, 'G:/EOData/temp/Meris_wew.dim', 'BEAM-DIMAP')

One word regarding the NDVI_OP. It is an implementation of an operator. You might understand it better if you read the doc provided at https://senbox.atlassian.net/wiki/display/SNAP/How+to+write+a+processor+in+Python.


Beginner Questions
Snappy SARSim-Terrain-Correction -- RuntimeError: java.lang.NullPointerException
#3

Thank you Marco!

What is java.util.HashMap? Would this be a list or array that contains strings/numerics of the parameters for the operator tools? Do I need to specify this or does the API know what is needed and have default values?

I really appreciate your help!


#4

HashMap is a dictionary. It maps parameter names (strings) to values (objects). It depends on the operator and parameter if a default value exists. Even if all parameters have a default value, an empty HashMap needs to be used.


#5

Note that if you put those operations in a single graph and run it, the performance will improve and disk-space requirements will diminish.


#6

If GPF is used from Python and the operators are concatenated it is like a single graph on the command line. No intermediate writing is performed.


#7

I stand corrected! I think I need to attend next time we organise a SNAP-training with python-content :slight_smile:


#8

Yes, would be nice. :slight_smile:


#9

Do these concatenated GPF operators need to be wrapped into a python class? Or is there any issue with a script such as:

import packages #snappy,Product,ProductIO,etc inFile='path/to/source' outFile=snappy.GPF.createProduct('Operator',parameters,inFile) ProductIO.writeProduct(outFile,'path/to/destination',......)

What advantage is there to placing these into a class as in snappy_glcm.py

Thank you!


#10

No you can use it like a script as you do in your example.
The advantage which @abgbaumann is showing by putting it into a class and a method is that it is better reusable in an other context.
What to use depends also on the use case you have.


#11

I’ve tried to concatenate GPF operations, but when I attempt to write out the file, I get an error. The ugly testing code is:

import snappy
from snappy import GPF
from snappy import ProductIO

params={}
inFile='/Radarsat2/quadpol_file.zip'
outDir='/test/directory'
outForm='BEAM-DIMAP'

HashMap = snappy.jpy.get_type('java.util.HashMap')
parameters = HashMap()
for a in params:
    parameters.put(a,params[a])
GPF.getDefaultInstance().getOperatorSpiRegistry().loadOperatorSpis()
product = snappy.ProductIO.readProduct(inFile)

Cal = GPF.createProduct('Calibration',parameters,product)
CalSf = GPF.createProduct('Polarimetric-Speckle-Filter',
                              parameters,Cal)
CalSfDec = GPF.createProduct('Polarimetric-Decomposition',
                                 parameters,CalSf)
CalSfDecTerr = GPF.createProduct('Terrain-Correction',
                                 parameters,CalSfDec)
ProductIO.writeProduct(CalSfDecTerr,outDir,outForm)

The error message is:

ProductIO.writeProduct(CalSfDecTerr,outDir,outForm)
Traceback (most recent call last):

  File "<ipython-input-30-3051e41d9fa0>", line 1, in <module>
    ProductIO.writeProduct(CalSfDecTerr,outDir,outForm)

RuntimeError: org.esa.snap.core.gpf.OperatorException: 4

What does this RuntimeError mean, and how can I fix it?

Also, does outDir in ProductIO.writeProduct need to be a path with filename, or just the path?

Thank you!


#12

Figure out that I hadn’t been including the parameter:
outputImageInComplex with true, which was causing the problem.

I am now having issues with JavaHeapSpace/DataBuffer, despite running the performance optimizing options. I’ll try modifying the snappy.ini as others have suggest.

I would still appreciate an answer regarding

Thank you!


#13

Hi,

the second paramter in ProductIO.writeProduct is the filename.


#14

Is it possible to run a built graph.xml by GPF? thx.


#15

Hi, @marpet, is it possible to run a pre-built graph.xml with GPF? and how should I do? thx.


#16

Sure. this is possible.
Just do:

gpt <Path_to_graph.xml> <Path_to_source_product>

You can also type

gpt -h

to get more help.
Or visit the documentation in our wiki:
https://senbox.atlassian.net/wiki/spaces/SNAP/pages/70503590/Creating+a+GPF+Graph
https://senbox.atlassian.net/wiki/spaces/SNAP/pages/70503475/Bulk+Processing+with+GPT


#17

@marpet Many thanks!
I want to use snappy for SAR interferogram, when I set parameters for TOPSAR-split, but I find it doesn’t work as expected.
could u help me to check it?

Set Params

params={
# TOPSAR-split
“subswath”: “IW2”,
“selectedPolarisations”: ‘VV’,
“bursts”: “1 to 2 (max number of bursts: 4)”, # not work
}

Parameters Assignments

parameters = HashMap()
for a in params:
# print(a)
parameters.put(a, params[a])

TOPSAR-Split

t1_spt = GPF.createProduct(“TOPSAR-Split”, parameters, product_t1)

Thanks in advance!


#18

I found some help with “gpt TOPSAR-Split -h”

Parameter Options:
-PfirstBurstIndex= The first burst index
Valid interval is [1, *).
Default value is ‘1’.
-PlastBurstIndex= The last burst index
Valid interval is [1, *).
Default value is ‘9999’.
-PselectedPolarisations=<string,string,string,…> The list of polarisations
-Psubswath= The list of source bands.
-PwktAoi= WKT polygon to be used for selecting bursts


#19

A ‘bursts’ parameter does not exist.
You need to use:

"firstBurstIndex":1
"lastBurstIndex":4

You can omit the burst indices. The they are calculated automatically.


#20

Thanks, it works now for selecting bursts, but the selected polarisations are not expected, always ‘VV’.

gpt Path_to_graph.xml Path_to_source_product

And, how should I run graph.xml from python scripts? The same as command line?