How to Process multiple files using C2RCC

binbinjiang · March 3, 2021, 3:52pm

try to follow your suggestion.

(binbin2) [bjiang@viz3]~/snap/bin% chmod u+x processDataset.bash resample_s2.xml resample_20m.properties “/home/bjiang/Sentinel2/getOC-master-3/2016woodram/orgin” “/home/bjiang/Sentinel2/getOC-master-3/2016woodram/orgin/S2A_MSIL1C_20160106T153632_N0201_R111_T19TCJ_20160106T153739.SAFE/C2RCC” resampled20m
chmod: cannot access ‘resampled20m’: No such file or directory
(binbin2) [bjiang@viz3]~/snap/bin% chmod u+x processDataset.bash
(binbin2) [bjiang@viz3]~/snap/bin% processDataset.bash resample_s2.xml resample_20m.properties “/home/bjiang/Sentinel2/getOC-master-3/2016woodram/orgin” “/home/bjiang/Sentinel2/getOC-master-3/2016woodram/orgin/S2A_MSIL1C_20160106T153632_N0201_R111_T19TCJ_20160106T153739.SAFE/C2RCC” resampled20m
zsh: processDataset.bash: command not found…
zsh: command not found: processDataset.bash

binbinjiang · March 3, 2021, 4:08pm

by the way, i fix the “processDataset.bash” for my gpt path, like below,

adapt this path to your needs

export PATH=~/home/bjiang/snap/bin:$PATH
gptPath=“gpt”

is that right? when i try it again, the error like this below
(binbin2) [bjiang@viz3]~/snap/bin% ./processDataset.bash resample_s2.xml resample_20m.properties “/home/bjiang/Sentinel2/getOC-master-3/2016woodram/orgin” “/home/bjiang/Sentinel2/getOC-master-3/2016woodram/orgin/S2A_MSIL1C_20160106T153632_N0201_R111_T19TCJ_20160106T153739.SAFE/C2RCC” resampled20m
./processDataset.bash: line 53: gpt: command not found
./processDataset.bash: line 53: gpt: command not found
./processDataset.bash: line 53: gpt: command not found
./processDataset.bash: line 53: gpt: command not found
./processDataset.bash: line 53: gpt: command not found
./processDataset.bash: line 53: gpt: command not found
./processDataset.bash: line 53: gpt: command not found
./processDataset.bash: line 53: gpt: command not found
./processDataset.bash: line 53: gpt: command not found
./processDataset.bash: line 53: gpt: command not found
./processDataset.bash: line 53: gpt: command not found
./processDataset.bash: line 53: gpt: command not found
./processDataset.bash: line 53: gpt: command not found
./processDataset.bash: line 53: gpt: command not found
./processDataset.bash: line 53: gpt: command not found
./processDataset.bash: line 53: gpt: command not found
./processDataset.bash: line 53: gpt: command not found
./processDataset.bash: line 53: gpt: command not found
./processDataset.bash: line 53: gpt: command not found
./processDataset.bash: line 53: gpt: command not found
./processDataset.bash: line 53: gpt: command not found
./processDataset.bash: line 53: gpt: command not found
./processDataset.bash: line 53: gpt: command not found
./processDataset.bash: line 53: gpt: command not found
./processDataset.bash: line 53: gpt: command not found
./processDataset.bash: line 53: gpt: command not found
./processDataset.bash: line 53: gpt: command not found
./processDataset.bash: line 53: gpt: command not found
./processDataset.bash: line 53: gpt: command not found
./processDataset.bash: line 53: gpt: command not found
./processDataset.bash: line 53: gpt: command not found
./processDataset.bash: line 53: gpt: command not found
./processDataset.bash: line 53: gpt: command not found
./processDataset.bash: line 53: gpt: command not found
./processDataset.bash: line 53: gpt: command not found
(binbin2) [bjiang@viz3]~/snap/bin%

Best

Binbin

binbinjiang · March 3, 2021, 8:41pm

It looks like works. thank you all. any suggestion for c2rcc atmospheric correction?

Best

Binbin

(binbin2) [bjiang@viz3]~/snap/bin% ./processDataset.bash resample_s2.xml resample_20m.properties “/home/bjiang/Sentinel2/getOC-master-3/2016woodram/trytryorgin” “/home/bjiang/Sentinel2/getOC-master-3/2016woodram/C2RCCtry” resampled20m
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: GDAL 3.0.2 found on system. JNI driver will be used.
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Installed GDAL 3.0.2 set to be used by SNAP.
Native library load failed.
java.lang.UnsatisfiedLinkError: /nfs3-home/bjiang/.snap/auxdata/gdal/gdal-3-0-X-jni/libgdalalljni.so: libgdal.so.26: cannot open shared object file: No such file or directory
SEVERE: org.esa.s2tbx.dataio.gdal.GDALLoader: Failed to initialize GDAL native drivers. GDAL readers and writers were disabled.java.lang.reflect.InvocationTargetException
INFO: org.esa.snap.core.util.EngineVersionCheckActivator: Please check regularly for new updates for the best SNAP experience.
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Installed GDAL 3.0.2 set to be used by SNAP.
Executing processing graph
INFO: org.esa.s2tbx.dataio.s2.ortho.S2OrthoProductReaderPlugIn: Building product reader - EPSG:32619
WARNING: org.esa.snap.core.metadata.GenericXmlMetadata: Metadata: the path to element [metadata_level] does not exist
WARNING: org.esa.snap.core.metadata.GenericXmlMetadata: Metadata: the path to element [granuleidentifier] does not exist
WARNING: org.esa.snap.core.metadata.GenericXmlMetadata: Metadata: the path to element [bandid] does not exist
INFO: org.hsqldb.persist.Logger: dataFileCache open start
…10%…20%…30%…40%…50%…60%…70%…80%…90% done.
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: GDAL 3.0.2 found on system. JNI driver will be used.
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Installed GDAL 3.0.2 set to be used by SNAP.
Native library load failed.
java.lang.UnsatisfiedLinkError: /nfs3-home/bjiang/.snap/auxdata/gdal/gdal-3-0-X-jni/libgdalalljni.so: libgdal.so.26: cannot open shared object file: No such file or directory
SEVERE: org.esa.s2tbx.dataio.gdal.GDALLoader: Failed to initialize GDAL native drivers. GDAL readers and writers were disabled.java.lang.reflect.InvocationTargetException
INFO: org.esa.snap.core.util.EngineVersionCheckActivator: Please check regularly for new updates for the best SNAP experience.
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Installed GDAL 3.0.2 set to be used by SNAP.
Executing processing graph
INFO: org.esa.s2tbx.dataio.s2.ortho.S2OrthoProductReaderPlugIn: Building product reader - EPSG:32619
WARNING: org.esa.snap.core.metadata.GenericXmlMetadata: Metadata: the path to element [metadata_level] does not exist
WARNING: org.esa.snap.core.metadata.GenericXmlMetadata: Metadata: the path to element [granuleidentifier] does not exist
WARNING: org.esa.snap.core.metadata.GenericXmlMetadata: Metadata: the path to element [bandid] does not exist
INFO: org.hsqldb.persist.Logger: dataFileCache open start
…10%…20%…30%…40%…50%…60%…70%…80%…90% done.
(binbin2) [bjiang@viz3]~/snap/bin%

marpet · March 4, 2021, 7:08am

Good that it now works.

But the chmod command should actually look like:

chmod u+x processDataset.bash

Not more. Afterwards it can be invoked the normal way.

binbinjiang · March 5, 2021, 4:17pm

Dear marpet,

after i get the sentinel2 20m , i want to c2rcc.msi process, but it not works. Could you have some suggestions? thanks

Best

Binbin
command line:
./gpt c2rcc.msi -SsourceProduct=’/home/bjiang/Sentinel2/getOC-master-3/2016woodram/C2RCCtry’ -PoutputRpath=’/home/bjiang/Sentinel2/getOC-master-3/2016woodram/C2RCCtry/C2RCC’
error
(binbin2) [bjiang@viz3]~/snap/bin% ./gpt c2rcc.msi -SsourceProduct=’/home/bjiang/Sentinel2/getOC-master-3/2016woodram/C2RCCtry’ -PoutputRpath=’/home/bjiang/Sentinel2/getOC-master-3/2016woodram/C2RCCtry/C2RCC’
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: GDAL 3.0.2 found on system. JNI driver will be used.
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Installed GDAL 3.0.2 set to be used by SNAP.
Native library load failed.
java.lang.UnsatisfiedLinkError: /nfs3-home/bjiang/.snap/auxdata/gdal/gdal-3-0-X-jni/libgdalalljni.so: libgdal.so.26: cannot open shared object file: No such file or directory
SEVERE: org.esa.s2tbx.dataio.gdal.GDALLoader: Failed to initialize GDAL native drivers. GDAL readers and writers were disabled.java.lang.reflect.InvocationTargetException
INFO: org.esa.snap.core.util.EngineVersionCheckActivator: Please check regularly for new updates for the best SNAP experience.
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Installed GDAL 3.0.2 set to be used by SNAP.
org.esa.snap.core.gpf.OperatorException: No product reader found for ‘/nfs3-home/bjiang/Sentinel2/getOC-master-3/2016woodram/C2RCCtry’
at org.esa.snap.core.gpf.main.DefaultCommandLineContext.readProduct(DefaultCommandLineContext.java:57)
at org.esa.snap.core.gpf.main.CommandLineTool.readProduct(CommandLineTool.java:534)
at org.esa.snap.core.gpf.main.CommandLineTool.addProduct(CommandLineTool.java:479)
at org.esa.snap.core.gpf.main.CommandLineTool.getSourceProductMap(CommandLineTool.java:467)
at org.esa.snap.core.gpf.main.CommandLineTool.runOperator(CommandLineTool.java:295)
at org.esa.snap.core.gpf.main.CommandLineTool.runGraphOrOperator(CommandLineTool.java:284)
at org.esa.snap.core.gpf.main.CommandLineTool.run(CommandLineTool.java:188)
at org.esa.snap.core.gpf.main.CommandLineTool.run(CommandLineTool.java:121)
at org.esa.snap.core.gpf.main.GPT.run(GPT.java:54)
at org.esa.snap.core.gpf.main.GPT.main(GPT.java:34)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.esa.snap.runtime.Launcher.lambda$run$0(Launcher.java:55)
at org.esa.snap.runtime.Engine.runClientCode(Engine.java:189)
at org.esa.snap.runtime.Launcher.run(Launcher.java:51)
at org.esa.snap.runtime.Launcher.main(Launcher.java:31)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.exe4j.runtime.LauncherEngine.launch(LauncherEngine.java:84)
at com.install4j.runtime.launcher.UnixLauncher.start(UnixLauncher.java:66)
at install4j.org.esa.snap.runtime.Launcher_gpt.main(Unknown Source)

Error: No product reader found for ‘/nfs3-home/bjiang/Sentinel2/getOC-master-3/2016woodram/C2RCCtry’
(binbin2) [bjiang@viz3]~/snap/bin%

marpet · March 7, 2021, 8:02am

The error explains it.
The path you have specified for the source product does not point to a product. But only to a directory, I guess.

binbinjiang · March 7, 2021, 7:52pm

Dear marpet,

how could fix the problem. thanks. i have no idea about it. the “C2RCCtry” directory already has the after bash resample 20m sentinel2 data “resampled20m_S2A_MSIL1C_20160106T153632.data” and “resampled20m_S2A_MSIL1C_20160106T153632.dim” . it should works with ./gpt c2rcc.msi .

Thanks.

Binbin

marpet · March 8, 2021, 7:33am

Have you modified the processDataset.bash file? You could compare it with the original file and maybe you see a difference.

binbinjiang · March 8, 2021, 12:27pm

Dear marpet,

thanks for your reply fast. How could i modified the processDataset.bash? could you provide an example? thanks a lot.

Best

Binbin

marpet · March 8, 2021, 12:47pm

Sorry, but I can’t provide more help which exceeds the original guide
Bulk Processing with GPT - SNAP
Maybe you start with the basics of scripting on unix. This might help.
Shell Scripting Tutorial: How to Create Shell Script in Linux/Unix (guru99.com)

marpet · March 17, 2023, 6:55am

A post was split to a new topic: How to get CDOM (Coloured Dissolved Organic Material) and DOC (Dissolved Organic Carbon)

cheesta · June 14, 2023, 6:54pm

Hi everyone, I have processed some Sentinel 3 OLCI data with the C2RCC processor. I was wondering if there is a way to average multiple images together using snap? Thank you.

marpet · June 15, 2023, 5:40am

Yes, you can use the mosaic processor or the binning processor.
The binning processor gives you more freedom on how the data aggregated. The mosaic only does a simple average.

cheesta · June 26, 2023, 3:13pm

Hello,
I have been trying to average a few images together using the level-3 binning processor but I keep getting this as the output image. I am not sure if my parameters are incorrect or if I am trying to open the image wrong, I am using the RGB image window.
Thank you,
Chloe
Screenshot 2023-06-26 105617

cheesta · June 26, 2023, 3:17pm

This image is also the size of very few pixels from the original images.

bidenntrumpp · June 28, 2023, 8:15am

I want to process multi MS1 scences in one go. I have exactly folowed the tutorial sent by Marco.

abruescas · June 28, 2023, 1:08pm

You will have to change the Spatial Resolution because your output has right now 9.28km pixel