Mosaic operator with subsequent BandMath operator to readd meta information to the bands

[scheunro]

Hello everyone

I’m looking to build a generic graph xml to overcome following shortcoming: SNAP-745. However it seems I need some help.
So far, I have only the Mosaic operator implemented: mosaic_.xml (869 Bytes)

Currently I handle the output of the attached graph with following Python code:

import os
from snappy import ProductIO
def copy_metadata(source_path, target_path):
    """ Adds spectral wavelength, unit, and valid pixel expression from the source product to the bands of the target product.
     Needed to postprocess the mosaic output: https://senbox.atlassian.net/browse/SNAP-745 """

    source_product = ProductIO.readProduct(source_path)
    target_product = ProductIO.readProduct(target_path)
    target_bands = target_product.getBandNames()
    temp_path = os.path.splitext(target_path)[0] + '_temp' + '.nc'

    for band in source_product.getBandNames():
        if band in target_bands:
            target_product.getBand(band).setSpectralWavelength(source_product.getBand(band).getSpectralWavelength())
            target_product.getBand(band).setUnit(source_product.getBand(band).getUnit())
            target_product.getBand(band).setValidPixelExpression(source_product.getBand(band).getValidPixelExpression())

    ProductIO.writeProduct(target_product, temp_path, 'NetCDF4-BEAM')
    os.remove(target_path)
    os.rename(temp_path, target_path)

My first question: is it even possible to this in a similarly general way as I do it in Python?
If yes: who knows how to?
If no: any alternative idea?

Thanks for you help in advance. It is much appreciated!

[marpet]

I’m not aware of another way to preserve the wavelength at the moment.
To simplify your processing chain you could call the mosaic also from Python. Then you have only one call.
How the mosaic could be used is shown here:

Mosaicing Sentinel-2 tiles in snappy

I’m trying to mosaic 2 Sentinel-2 tiles using snappy but when I execute this code: products = jpy.array('org.esa.snap.core.datamodel.Product', 2) products[0] = ProductIO.readProduct(sourcePath1) products[1] = ProductIO.readProduct(sourcePath2) bnames = products[0].getBandNames() #Bands are the same for each product parameters = HashMap() parameters.put('variables',bnames) Mosaic = GPF.createProduct('Mosaic', parameters, products) I get this error: RuntimeError: org.esa.snap.core.gpf.…

Or you could execute the Graph XML file as I’ve explained here:

Example script for multiple operations?

In python you don’t need the XML. here you can use the output of one operator as in input to the next operator. You can start command line sub-processes to start the processing with gpt, or execute the graph directly as in the following snippet: import snappy from snappy import jpy, ProgressMonitor FileReader = jpy.get_type('java.io.FileReader') GraphIO = jpy.get_type('org.esa.snap.core.gpf.graph.GraphIO') Graph = jpy.get_type('org.esa.snap.core.gpf.graph.Graph') GraphProcessor = jpy.get_type…

See also what the user @xidiandreamseeker wrote further down the thread:

Example script for multiple operations?

Thanks for your great help, I finally implement batch processing by runing graph from python. and I set read and write nodes as follows: graph.getNode(“read”).getConfiguration().getChild(0).setValue(String input1) graph.getNode(“read(2)”).getConfiguration().getChild(0).setValue(String input2) graph.getNode(‘write’).getConfiguration().getChild(0).setValue(String write)

[scheunro]

Hi @marpet,

Thank you for you quick response!
My idea was to the opposite: do everything in the Graph XML and do not use snappy. This would give me two advantages: first, better reproducibility (I save the Graph XML for each result), secondly, simpler installation procedure as snappy and jpy are not that straight forward to install.

So one idea that comes to my mind is to use Bandmath operator for every band I want the meta information, however, I guess I would have to explicitly specify every band in the Graph XML?

[marpet]

Oh yes, with BandMaths this is possible. Sorry I forgot. I have suggested it some time ago .

Gpt Mosaic of multiple input files

You can add the wavelength in the description of the target bands in the BandMaths operator. The operator supports multiple target bands. <targetBands> <targetBand> <name>string</name> <type>string</type> <expression>string</expression> <description>string</description> <unit>string</unit> <validExpression>string</validExpression> <noDataValue>double</noDataValue> <spectralBandIndex>…

You don’t need to specify all bands, only the wavelength bands. In a subsequent step you can use the Merge operator. There you can merge your original mosaic (exclude the wavelength bands) with the BandMaths output.

But it might be simpler if copy all bands in the BandMaths operation.

[scheunro]

How can I copy all bands in the BandMaths operation?

[marpet]

You need to add for every band a targetband section

<targetBand>
  <name>string</name>
  <type>string</type>
  <expression>string</expression>
  <description>string</description>
  <unit>string</unit>
  <validExpression>string</validExpression>
  <noDataValue>double</noDataValue>
  <spectralBandIndex>integer</spectralBandIndex>
  <spectralWavelength>float</spectralWavelength>
  <spectralBandwidth>float</spectralBandwidth>
  <scalingOffset>double</scalingOffset>
  <scalingFactor>double</scalingFactor>
</targetBand>

Leave out the properties you don’t want to set.
A simple case might look like:

<targetBand>
  <name>myBand</name>
  <type>float32</type>
  <expression>myBand</expression>
  <description>Description of my band</description>
  <unit>1/m^3</unit>
</targetBand>

[scheunro]

Ok, now I have problems merging the BandMaths output with the mosaicing output:

INFO: org.esa.snap.core.datamodel.Product: first scan line left corner org.esa.snap.core.datamodel.GeoPos[47�53'19" N,5�48'07" E] not equal to org.esa.snap.core.datamodel.GeoPos[47�53'24" N,5�48' E]
org.esa.snap.core.gpf.graph.GraphException: [NodeId: mergeNode] Product [bandmathsProduct] is not compatible to master product.
	at org.esa.snap.core.gpf.graph.NodeContext.initTargetProduct(NodeContext.java:79)
	at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:199)
	at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:182)
	at org.esa.snap.core.gpf.graph.GraphContext.initOutput(GraphContext.java:166)
	at org.esa.snap.core.gpf.graph.GraphContext.<init>(GraphContext.java:85)
	at org.esa.snap.core.gpf.graph.GraphContext.<init>(GraphContext.java:58)
	at org.esa.snap.core.gpf.graph.GraphProcessor.executeGraph(GraphProcessor.java:118)
	at org.esa.snap.core.gpf.main.DefaultCommandLineContext.executeGraph(DefaultCommandLineContext.java:86)
	at org.esa.snap.core.gpf.main.CommandLineTool.executeGraph(CommandLineTool.java:547)
	at org.esa.snap.core.gpf.main.CommandLineTool.runGraph(CommandLineTool.java:391)
	at org.esa.snap.core.gpf.main.CommandLineTool.runGraphOrOperator(CommandLineTool.java:287)
	at org.esa.snap.core.gpf.main.CommandLineTool.run(CommandLineTool.java:188)
	at org.esa.snap.core.gpf.main.CommandLineTool.run(CommandLineTool.java:121)
	at org.esa.snap.core.gpf.main.GPT.run(GPT.java:60)
	at org.esa.snap.core.gpf.main.GPT.main(GPT.java:37)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.esa.snap.runtime.Launcher.lambda$run$0(Launcher.java:55)
	at org.esa.snap.runtime.Engine.runClientCode(Engine.java:189)
	at org.esa.snap.runtime.Launcher.run(Launcher.java:51)
	at org.esa.snap.runtime.Launcher.main(Launcher.java:31)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.exe4j.runtime.LauncherEngine.launch(LauncherEngine.java:84)
	at com.exe4j.runtime.WinLauncher.main(WinLauncher.java:94)
	at com.install4j.runtime.launcher.WinLauncher.main(WinLauncher.java:25)
Caused by: org.esa.snap.core.gpf.OperatorException: Product [bandmathsProduct] is not compatible to master product.
	at org.esa.snap.core.gpf.common.MergeOp.validateSourceProducts(MergeOp.java:223)
	at org.esa.snap.core.gpf.common.MergeOp.initialize(MergeOp.java:105)
	at org.esa.snap.core.gpf.internal.OperatorContext.initializeOperator(OperatorContext.java:528)
	at org.esa.snap.core.gpf.internal.OperatorContext.getTargetProduct(OperatorContext.java:298)
	at org.esa.snap.core.gpf.Operator.getTargetProduct(Operator.java:385)
	at org.esa.snap.core.gpf.graph.NodeContext.initTargetProduct(NodeContext.java:77)
	... 29 more

Error: [NodeId: mergeNode] Product [bandmathsProduct] is not compatible to master product.

Processing graph: idepix_OLCI.xml (2.3 KB)
Mosaicing graph: mosaic_20200617.xml (19.8 KB)

I use following to products:

• S3B_OL_1_EFR____20200617T101449_20200617T101749_20200618T144129_0179_040_122_2160_LN1_O_NT_002.SEN3
• S3A_OL_1_EFR____20200617T091317_20200617T091617_20200618T141059_0179_059_264_2160_LN1_O_NT_002.SEN3

[scheunro]

I understand the error message, but how to resolve that problem? It seems that the mosaicNode adds a slight shift to the product. I mean the difference of 7" is quite small.

[marpet]

Could be that the is slight difference how the reprojection is applied in the Reprojection operation and in the mosaic.
As long if both input which shall be merged have the same dimension and you know that they cover the same region you can set the geographicError parameter.
Add this parameter section

<parameters>
  <geographicError>NaN</geographicError>
</parameters>

This defines the maximum lat/lon error in degree between the products. If set to NaN no check is performed.

[scheunro]

Perfect, this did the trick!

Do you have an idea how I could omit explicitly specifying every single band?

[scheunro]

Using multi-size mosaic seems to also help in my case. As suggested here:
After Mosaic data loss spectral information

Using this I do not need the BandMaths operator and therefore I do not need to specify every single band.