Rad2Refl after Rayleigh correction

Hi everyone,

I am trying to convert Rayleigh corrected radiances to reflectances from S3 L1 data.

I was progressing well till I got the not-so-useful following log “Error: org.esa.snap.core.gpf.graph.GraphException”. I didn’t find a way to get a deeper log, so I’m currently stuck trying to debug this graph… Any help would be more than welcome!

myGraph.xml (2.7 KB)

I am running SNAP 9.0.0 on ubuntu 22.04.1.

Thank you a lot in advance,
best,
Adrien

Hello Adrien,

The error is probably caused by incompatibility of the Rayleigh and the Rad2Refl operator.
You can check the log file for more information: How to find the log files?
The Rad2Refl expects original radiances and the rayleigh creates bottom-of-rayleigh reflectances.
So, I think you can simply remove the Radiance-to-Reflectance step.

Hello @marpet,

thank you a lot for your fast answer and help, that makes sense!

I’m therefore now using a graph you shared here, applied to my case:

Olci_Rayleigh.xml (805 Bytes)

That I call with the following in a short bash script:

xml=/home/adrien/Downloads/Olci_Rayleigh.xml

olci_folder=/home/adrien/Downloads/S3A_OL_1_EFR____20190801T175136_20190801T175436_20190802T225244_0179_047_312_1620_LN1_O_NT_002.SEN3

dest=/home/adrien/Downloads/S3A_OL_1_EFR____20190801T175136_20190801T175436_20190802T225244_0179_047_312_1620_LN1_O_NT_002.SEN3

sudo ~/snap/snap/bin/gpt ${xml} -POLCIsource="${olci_folder}" -PtargetFolder="${dest}" -Dsnap.log.level=ERROR -e

Unfortunately the source is not read properly, and I don’t really see where to go from here…

INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.esa.snap.core.util.EngineVersionCheckActivator: Please check regularly for new updates for the best SNAP experience.
Executing processing graph
 done.
org.esa.snap.core.gpf.graph.GraphException: Missing source '/home/adrien/Downloads/S3A_OL_1_EFR____20190801T175136_20190801T175436_20190802T225244_0179_047_312_1620_LN1_O_NT_002.SEN3/xfdumanifest.xml' in node 'rayleigh'
	at org.esa.snap.core.gpf.graph.GraphContext.initNodeDependencies(GraphContext.java:107)
	at org.esa.snap.core.gpf.graph.GraphContext.<init>(GraphContext.java:84)
	at org.esa.snap.core.gpf.graph.GraphContext.<init>(GraphContext.java:58)
	at org.esa.snap.core.gpf.graph.GraphProcessor.executeGraph(GraphProcessor.java:127)
	at org.esa.snap.core.gpf.main.DefaultCommandLineContext.executeGraph(DefaultCommandLineContext.java:86)
	at org.esa.snap.core.gpf.main.CommandLineTool.executeGraph(CommandLineTool.java:547)
	at org.esa.snap.core.gpf.main.CommandLineTool.runGraph(CommandLineTool.java:391)
	at org.esa.snap.core.gpf.main.CommandLineTool.runGraphOrOperator(CommandLineTool.java:287)
	at org.esa.snap.core.gpf.main.CommandLineTool.run(CommandLineTool.java:188)
	at org.esa.snap.core.gpf.main.CommandLineTool.run(CommandLineTool.java:121)
	at org.esa.snap.core.gpf.main.GPT.run(GPT.java:59)
	at org.esa.snap.core.gpf.main.GPT.main(GPT.java:37)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.esa.snap.runtime.Launcher.lambda$run$0(Launcher.java:55)
	at org.esa.snap.runtime.Engine.runClientCode(Engine.java:189)
	at org.esa.snap.runtime.Launcher.run(Launcher.java:51)
	at org.esa.snap.runtime.Launcher.main(Launcher.java:31)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.exe4j.runtime.LauncherEngine.launch(LauncherEngine.java:84)
	at com.install4j.runtime.launcher.UnixLauncher.start(UnixLauncher.java:66)
	at install4j.org.esa.snap.runtime.Launcher_gpt.main(Unknown Source)

Error: Missing source '/home/adrien/Downloads/S3A_OL_1_EFR____20190801T175136_20190801T175436_20190802T225244_0179_047_312_1620_LN1_O_NT_002.SEN3/xfdumanifest.xml' in node 'rayleigh'

Thank you a lot for your help!

best,
Adrien

That’s because source products and other parameters are treated differently.
An operator expects already a product instance as parameter in the sources section.
In your case it is only a string parameter (the path).

You have two options to fix it.
You can add Read operator beforehand.
This accepts the path as parameter ‘file’.

<node id="someNodeId">
      <operator>Read</operator>
      <sources/>
      <parameters>
        <file>file</file>
        <formatName>string</formatName>
        <sourceBands>string,string,string,...</sourceBands>
        <sourceMasks>string,string,string,...</sourceMasks>
        <pixelRegion>rectangle</pixelRegion>
        <geometryRegion>geometry</geometryRegion>
        <useAdvancedOptions>boolean</useAdvancedOptions>
        <copyMetadata>boolean</copyMetadata>
      </parameters>
    </node>

In this case you can further use the string. and the ‘-P’-option on the command line.

The simpler solution is to switch to the ‘-S’-option. This tells gpt to read a product from the path.
In this case you need to move the xfdumanifest.xml file to the script or you completely remove it.
The path to the directory works too.
Example of the cmd call (for Windows, sorry) is provided.
gpt "E:\_UserSupport\AdrienWehrle\Olci_Rayleigh.xml" -SOLCIsource=E:\SENTINEL3\OLCI\S3A_OL_1_EFR____20190214T070944_20190214T071244_20190215T112348_0179_041_220_2160_LN1_O_NT_002.SEN3 -Pdest=E:\_temp -Dsnap.log.level=ERROR -e
(contained in the header of the update graph too)

Olci_Rayleigh.xml (1.0 KB)

Thank you a lot @marpet, option 2 works like a charm! Only thing is I would like to save the Rayleigh corrected reflectances to tif files with the Write operator but don’t know what sourceProduct to point to after the correction? I suppose I need to create subsets beforehand with the Subset operator anyways, but still, I don’t find the name of the source band after correction… Thank you a lot for your help!

Probably this post can help you:

To get the names you should do the processing once and write the product. Then you can open it in SNAP and inspect the result. Or you do the processing directly in SNAP. The bands you are looking for are named “rBRR_##”
As source product you can specify the name of the node which shall provide the input.
This is also shown in the thread above.

The bands you are looking for are named “rBRR_##”

Perfect, thank you so much!

Hi again @marpet,

I followed your examples and produced the following graph:

OLCI_Rayleigh_correction.xml (1.5 KB)

which gives me

Error: The Product 'Subset_projected_S3A_OL_1_EFR____20190801T175136_20190801T175436_20190802T225244_0179_047_312_1620_LN1_O_NT_002.SEN3_rayleigh' already contains a raster data node with the name 'rBRR_10'.

I tried many different combinations, with and without subset, trying to write directly after projection since apparently the subset is already there… But somehow can’t make it work so far. Do you see what I do wrong here?

Thank you so much for your help!

Here the issue is that the name of the file name must not be equal to one of the bands.

Within a product no duplication of node names is allowed. When specifying the file name, it is used as name for the product, and this results in the duplication of the names.
Change it to ‘OLCI_rBRR_10.tif’ then it works.

I’ve added a ticket for a better error reporting.
[SNAP-1575] Improve error report when file name duplicate node name for GeoTiff - JIRA (atlassian.net)

1 Like

Thank you a lot for your help @marpet, this indeed works perfectly now! A ticket for a better error reporting sounds like a good idea, it was very hard to trace back to the actual error source IMO…!

Have a nice day,
Adrien