Snappy: Collocate, set 'masterProduct'

Hello

I am trying to implement Collocate into a Python Script.

trg_data = GPF.createProduct(‘Collocate’, parameters, src_data)

I get the following error:

RuntimeError: org.esa.snap.core.gpf.OperatorException: Operator ‘CollocateOp’: Mandatory source product (field ‘masterProduct’) not set.

Does anyone know how to set this field?

Similar to the parameters you can create a map for the source products and then call GPF

sourceProducts = HashMap()
sourceProducts.put("master", mProduct)
sourceProducts.put("slave", sProduct)
GPF.createProduct('Collocate', parameters, sourceProducts)
2 Likes

Again thank you very much for your help! It works.

What is mProduct and sProduct here? I mean, I have 15 images , then how will I define mProduct and sProduct?

The example is from the times were on two products could be collocated in one step.
Now, you can collocate more products and there are more options. This can make it more complicated or easier.
If all your products cover already the same region, and you just want to create a stack then you can leave out the master/slave parameters.
Instead you can simply add a comma separated list of all products paths to the parameters map.

parameters.put("sourceProductPaths", pathsList)
GPF.createProduct('Collocate', parameters)

If you want to define a master, the one which defines the geographical extent, then find this one in SNAP desktop and get the name of it.
Right-click on it in the Product Explorer and select Properties. Here the name is shown and you can copy it.
Set this name as parameter ‘masterProductName’.

parameters.put("sourceProductPaths", pathsList)
parameters.put("masterProductName", 'THE_NAME_OF_THE_PRODUCT')
GPF.createProduct('Collocate', parameters)

This should do it.

I will follow up this question with another of mine, since I have a similar/related doubt.
I’ve been processing Sentinl-1 for a while, but recently I found that none of the processed images align with each other.
E.g. I have 60 Sentinel-1 images of one area from one year from the same orbit. I didn’t use ApplyOrbitFile step as I would like to use the image as soon as it has been acquired and not wait 20 days for it (as I saw in other discussions).
For the sake of my research (crop classification) I want images to align, pixel to pixel between all the images.
Would collocation be a good solution? What is a good way to implement it in processing workflow that is performed for each image individually? I would like to keep each image independently, not as a stack.
In case ApplyPreciseOrbit step was included in workflow, would the processed images align or still no?

P.S. Good to hear that more than two product can be collocated at once.

As this question is very radar related, maybe @ABraun you can make a suggestion what to do here?

My images :

My code :

coreg_set = [’/home/ubuntu/SOIL_MOISTURE/PROCESSED_DATA/S1A_IW_GRDH_1SDV_20200103T003500_20200103T003525_030629_038272_70BC_orbit_border_thermal_calib_subset_terrain_subset.dim’,
‘/home/ubuntu/SOIL_MOISTURE/PROCESSED_DATA/S1A_IW_GRDH_1SDV_20200115T003459_20200115T003524_030804_03888F_9715_orbit_border_thermal_calib_subset_terrain_subset.dim’,
‘/home/ubuntu/SOIL_MOISTURE/PROCESSED_DATA/S1A_IW_GRDH_1SDV_20200127T003459_20200127T003524_030979_038EBA_1C78_orbit_border_thermal_calib_subset_terrain_subset.dim’,
‘/home/ubuntu/SOIL_MOISTURE/PROCESSED_DATA/S1A_IW_GRDH_1SDV_20200208T003459_20200208T003524_031154_0394D7_C79C_orbit_border_thermal_calib_subset_terrain_subset.dim’,
‘/home/ubuntu/SOIL_MOISTURE/PROCESSED_DATA/S1A_IW_GRDH_1SDV_20200220T003458_20200220T003523_031329_039AE1_97D5_orbit_border_thermal_calib_subset_terrain_subset.dim’]

parameters = HashMap()
parameters.put(“sourceProductPaths”, coreg_set)
coll_img = GPF.createProduct(‘Collocate’, parameters)

Forget about terrain in file name. Terrain correction is not applied, only naming is added by mistake.

Getting the following error:

ValueError Traceback (most recent call last)
in
13 ‘/home/ubuntu/SOIL_MOISTURE/PROCESSED_DATA/S1A_IW_GRDH_1SDV_20200220T003458_20200220T003523_031329_039AE1_97D5_orbit_border_thermal_calib_subset_terrain_subset.dim’]
14 parameters = HashMap()
—> 15 parameters.put(“sourceProductPaths”, coreg_set)
16 coll_img = GPF.createProduct(‘Collocate’, parameters)
17

ValueError: cannot convert a Python ‘list’ to a Java ‘java.lang.Object’

Try just a string not a python list.
Like:

’/home/ubuntu/SOIL_MOISTURE/PROCESSED_DATA/S1A_IW_GRDH_1SDV_20200103T003500_20200103T003525_030629_038272_70BC_orbit_border_thermal_calib_subset_terrain_subset.dim,/home/ubuntu/SOIL_MOISTURE/PROCESSED_DATA/S1A_IW_GRDH_1SDV_20200115T003459_20200115T003524_030804_03888F_9715_orbit_border_thermal_calib_subset_terrain_subset.dim,/home/ubuntu/SOIL_MOISTURE/PROCESSED_DATA/S1A_IW_GRDH_1SDV_20200127T003459_20200127T003524_030979_038EBA_1C78_orbit_border_thermal_calib_subset_terrain_subset.dim’

CODE:

coreg_set = ‘/home/ubuntu/SOIL_MOISTURE/PROCESSED_DATA/S1A_IW_GRDH_1SDV_20200103T003500_20200103T003525_030629_038272_70BC_orbit_border_thermal_calib_subset_terrain_subset.dim, /home/ubuntu/SOIL_MOISTURE/PROCESSED_DATA/S1A_IW_GRDH_1SDV_20200115T003459_20200115T003524_030804_03888F_9715_orbit_border_thermal_calib_subset_terrain_subset.dim’

parameters = HashMap()
parameters.put(“sourceProductPaths”, coreg_set)
coll_img = GPF.createProduct(‘Collocate’, parameters)

ERROR:

RuntimeError Traceback (most recent call last)
in
19 parameters.put(“sourceProductPaths”, coreg_set)
20 parameters.put(“resamplingType”,‘NEAREST_NEIGHBOUR’)
—> 21 coll_img = GPF.createProduct(‘Collocate’, parameters)
22
23 band_names = coll_img.getBandNames()

RuntimeError: java.lang.NullPointerException

Hi @janardanroy,

Was wondering if you managed to resolve the null pointer exception error?
I am also facing the same issue here and would appreciate any insights.

My code is as follow:
# Read the product name of Sen1 data so that we can set it as the master
s1_product = ProductIO.readProduct(sen1)
s1_product_name = s1_product.getName()

    parameters = HashMap()
    parameters.put('sourceProductPaths', path_str)
    parameters.put('masterProductName', s1_product_name)
    parameters.put('renameMasterComponents', 'false')
    parameters.put('renameSlaveComponents', 'false')
    parameters.put('resamplingType', 'NEAREST_NEIGHBOUR')

    colocated = GPF.createProduct('Collocate', parameters)
2 Likes

Hi @xholmes ,
I am facing your same problem. Have you found a solution for this? @marpet please, could you kindly help us on this. I have followed your advice above (Snappy: Collocate, set 'masterProduct' - #5 by marpet) but when I run the last line GPF.createProduct('Collocate', parameters), I got the same error message like @xholmes .
Thank you in advance for your help.

S Savastano

@s.savastano I used CreateStack instead of collocate and it worked well.
I used the following:

coreg

1 Like

Dear @janardanroy
thank you for your suggestion.
Do you know if the output from CreateStack is the same of Collocate? I have a long series of Landsat images and I am analyzing the change in a specific AOI. For this, I need an alignment among all the pixels in the scene.
Thank you in advance.

@s.savastano I also used this for a long time series (4 years) of Sentinel-1 data. Output was promising in terms of pixel alignment.

1 Like

“RuntimeError: java.lang.NullPointerException” basically means an error has occurred. Many different problems will generate the same message, so it is very possilbe that your problem is not related to this topic.

You should provide enough detail to allow others to reproduce the problem, including your SNAP version (is it fully updated?), input data, and python script.

Note that many overly helpful editors replace ASCII characters like quotes and dashes with “look-alike” unicode glyphs. This is so common and hard to detect visually that many editors (emacs, MS Code) provide ways to check for non-ASCII characters in your script.

Hi @s.savastano,

Liked @janardanroy , i also switched to using CreateStack. However, to answer your question, I recalled the issue for me was primarily because of how the “sourceProductPaths” was being defined. I cannot remember which is the correct one, but I think the variable you pass in to that parameter has to be a list.

It has been sometime and I do not have the code that worked for me, so apologies if that is not the solution.

Hope this helps.

1 Like

Dear @gnwiii ,
thank you for your reply.
Yes SNAP is updated to the last version.
I would like to collocate a list of Landsat images (more than 200) included in a folder.
To do that, I have created a simple python function, using the suggestions from @marpet above

import snappy
snappy.GPF.getDefaultInstance().getOperatorSpiRegistry().loadOperatorSpis()
HashMap = snappy.jpy.get_type("java.util.HashMap")

def collocation(pathsList):
    parameters = HashMap()
    parameters.put("sourceProductPaths", pathsList)
    parameters.put("masterProductName",'LC08_L2SP_224084_20201126_20210316_02_T1_NDVI')
    parameters.put("targetProductName", 'collocated')
    parameters.put("targetProductType", 'COLLOCATED')
    parameters.put("renameMasterComponents", True)
    parameters.put("renameSlaveComponents", True)
    parameters.put("masterComponentPattern", '${ORIGINAL_NAME}_M')
    parameters.put("slaveComponentPattern", '${ORIGINAL_NAME}_S')
    parameters.put("resamplingType", 'NEAREST_NEIGHBOUR')
    output = snappy.GPF.createProduct('Collocate', parameters)
    return output

So, supposing that the folder contains just two tif files

  1. LC08_L2SP_224084_20201126_20210316_02_T1_NDVI.tif (master)
  2. LC08_L2SP_224084_20210606_20210615_02_T1_NDVI.tif (slave)

I give in input to the script the following pathsList as suggested by @marpet
pathsList = ('D:\\isardSAT\\coastal-change\\buenos_aires\\Output\\224084\\Low_Tide\\NDVI\\LC08_L2SP_224084_20201126_20210316_02_T1_NDVI.tif,D:\\isardSAT\\coastal-change\\buenos_aires\\Output\\224084\\Low_Tide\\NDVI\\LC08_L2SP_224084_20210606_20210615_02_T1_NDVI.tif')

and then that error message comes out.
It is important to highlight that this error doesn’t happen when I use the same files through the graph builder with this xml

<graph id="Graph">
  <version>1.0</version>
  <node id="Read">
    <operator>Read</operator>
    <sources/>
    <parameters class="com.bc.ceres.binding.dom.XppDomElement">
      <file>D:\isardSAT\coastal-change\buenos_aires\Output\224084\Low_Tide\NDVI\LC08_L2SP_224084_20201126_20210316_02_T1_NDVI.tif</file>
    </parameters>
  </node>
  <node id="Read(2)">
    <operator>Read</operator>
    <sources/>
    <parameters class="com.bc.ceres.binding.dom.XppDomElement">
      <file>D:\isardSAT\coastal-change\buenos_aires\Output\224084\Low_Tide\NDVI\LC08_L2SP_224084_20210606_20210615_02_T1_NDVI.tif</file>
    </parameters>
  </node>
  <node id="Collocate">
    <operator>Collocate</operator>
    <sources>
      <sourceProduct refid="Read"/>
      <sourceProduct.1 refid="Read(2)"/>
    </sources>
    <parameters class="com.bc.ceres.binding.dom.XppDomElement">
      <sourceProductPaths/>
      <masterProductName>LC08_L2SP_224084_20201126_20210316_02_T1_NDVI</masterProductName>
      <targetProductName>_collocated</targetProductName>
      <targetProductType>COLLOCATED</targetProductType>
      <renameMasterComponents>true</renameMasterComponents>
      <renameSlaveComponents>true</renameSlaveComponents>
      <masterComponentPattern>${ORIGINAL_NAME}_M</masterComponentPattern>
      <slaveComponentPattern>${ORIGINAL_NAME}_S</slaveComponentPattern>
      <resamplingType>NEAREST_NEIGHBOUR</resamplingType>
    </parameters>
  </node>
  <node id="Write">
    <operator>Write</operator>
    <sources>
      <sourceProduct refid="Collocate"/>
    </sources>
    <parameters class="com.bc.ceres.binding.dom.XppDomElement">
      <file>D:\isardSAT\coastal-change\buenos_aires\test\_collocated.dim</file>
      <formatName>BEAM-DIMAP</formatName>
    </parameters>
  </node>
  <applicationData id="Presentation">
    <Description/>
    <node id="Read">
            <displayPosition x="56.0" y="50.0"/>
    </node>
    <node id="Read(2)">
      <displayPosition x="58.0" y="176.0"/>
    </node>
    <node id="Collocate">
      <displayPosition x="287.0" y="124.0"/>
    </node>
    <node id="Write">
            <displayPosition x="455.0" y="135.0"/>
    </node>
  </applicationData>
</graph>


It is also true that changing the python script and including the two sources (master and slave) as a new Hashmap as indicated here


def collocation(master, slave):
     sources = HashMap()
     sources.put('master', master)
     sources.put('slave', slave)
     parameters = HashMap()
     parameters.put("targetProductName", 'collocated')
     parameters.put("targetProductType", 'COLLOCATED')
     parameters.put('renameMasterComponents', True)
     parameters.put('renameSlaveComponents', True)
     parameters.put('masterComponentPattern', '${ORIGINAL_NAME}_M')
     parameters.put('slaveComponentPattern', '${ORIGINAL_NAME}_S')
     parameters.put('resamplingType', 'NEAREST_NEIGHBOUR')
     collocated = snappy.GPF.createProduct('Collocate', parameters, sources)
     return output


where
master= ProductIO.readProduct('D:\\isardSAT\\coastal-change\\buenos_aires\\Output\\224084\\Low_Tide\\NDVI\\LC08_L2SP_224084_20201126_20210316_02_T1_NDVI.tif')
and
slave= ProductIO.readProduct('D:\\isardSAT\\coastal-change\\buenos_aires\\Output\\224084\\Low_Tide\\NDVI\\LC08_L2SP_224084_20210606_20210615_02_T1_NDVI.tif')
``
Thank you in advance for any suggestions.

S Savastano