Empy result on c2rcc for Sentinel2

[sentinel]

Hi,

I am processing a S2 1C product with C2RCC to derive water quality parameters. I am running the process via GPT. Before running the c2rcc operator, I have resampled and masked to my AOI the product of interest.

After that, I run this commnad:

gpt c2rcc.msi -SsourceProduct=path\S2B_MSIL1C_20220818T095549_N0400_R122_T33UXS_20220818T104434_rsp_subset.dim -p path\c2rcc_param.xml -t path\S2B_MSIL1C_20220818T095549_N0400_R122_T33UXS_20220818T104434_c2rcc

The process seems to run fine as I dont see any error on the cmd. However, when I open the products, they seem empty:

Any idea what am I missing? It happens to me doing it via GPT and GUI

Maybe this is related to this bug reported here:

C2RCC-MSI for S2 Not Producing Output Since Jan 14

Hi Marco … is there any update on this?

@marpet @abruescas

[abruescas]

I have tried and succeeded with a product from May, first subsetting and later resampling, via GUI:
subset_0_of_S2A_MSIL1C_20220512T095031_N0400_R079_T33TYN_20220512T115002_resample
It seems ok so far. I will check now with gpt.

[sentinel]

@abruescas thanks for the feedback. To discard issues, could you please try with the same productID:
S2B_MSIL1C_20220818T095549_N0400_R122_T33UXS_20220818T104434

And the same subset area:
POLYGON ((17.49745865334892 50.8539698216942, 17.49745865334892 51.247057044051104, 16.76719639371271 51.247057044051104, 16.76719639371271 50.8539698216942, 17.49745865334892 50.8539698216942))

The preprocessing I did with a graph via GPT where I resampled to 10m and then subset. The output of that was then the input of C2RCC in two tests. Running via GPT and GUI. Both with empty results. Below the parameters I used:

[abruescas]

This worked for me. But I cannot see water masses in this area, only rivers, is that right?
S2Resampling_subset_c2rcc_msi.xml (2.6 KB)

By the way, I did not use the standard valid pixel expression, it gave some errors (still trying to figure out why). And I work in Mac.

[sentinel]

Indeed the focus of the AOI is a river not a big water body. I have run your graph and I am facing this issue:

INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: GDAL not found on system. Internal GDAL 3.0.0 from distribution will be used. (f1)
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Internal GDAL 3.0.0 set to be used by SNAP.
INFO: org.esa.snap.core.util.EngineVersionCheckActivator: Please check regularly for new updates for the best SNAP experience.
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Internal GDAL 3.0.0 set to be used by SNAP.
Executing processing graph
INFO: org.esa.s2tbx.dataio.s2.ortho.S2OrthoProductReaderPlugIn: Building product reader - EPSG:32633
WARNING: org.esa.snap.core.metadata.GenericXmlMetadata: Metadata: the path to element [metadata_level] does not exist
WARNING: org.esa.snap.core.metadata.GenericXmlMetadata: Metadata: the path to element [granuleidentifier] does not exist
WARNING: org.esa.snap.core.metadata.GenericXmlMetadata: Metadata: the path to element [bandid] does not exist
INFO: org.hsqldb.persist.Logger: dataFileCache open start
WARNING: org.esa.s2tbx.dataio.s2.ortho.S2OrthoUtils: Warning: impossible to read properly mypath\S2B_MSIL1C_20220818T095549_N0400_R122_T33UXS_20220818T104434.SAFE\GRANULE\L1C_T33UXS_A028462_20220818T095747\QI_DATA\MSK_DETFOO_B01.jp2.gml

WARNING: org.esa.s2tbx.dataio.s2.ortho.S2OrthoUtils: Warning: impossible to read properly mypath\S2B_MSIL1C_20220818T095549_N0400_R122_T33UXS_20220818T104434.SAFE\GRANULE\L1C_T33UXS_A028462_20220818T095747\QI_DATA\MSK_DETFOO_B02.jp2.gml

WARNING: org.esa.s2tbx.dataio.s2.ortho.S2OrthoUtils: Warning: impossible to read properly mypath\S2B_MSIL1C_20220818T095549_N0400_R122_T33UXS_20220818T104434.SAFE\GRANULE\L1C_T33UXS_A028462_20220818T095747\QI_DATA\MSK_DETFOO_B03.jp2.gml

WARNING: org.esa.s2tbx.dataio.s2.ortho.S2OrthoUtils: Warning: impossible to read properly mypath\S2B_MSIL1C_20220818T095549_N0400_R122_T33UXS_20220818T104434.SAFE\GRANULE\L1C_T33UXS_A028462_20220818T095747\QI_DATA\MSK_DETFOO_B04.jp2.gml

WARNING: org.esa.s2tbx.dataio.s2.ortho.S2OrthoUtils: Warning: impossible to read properly mypath\S2B_MSIL1C_20220818T095549_N0400_R122_T33UXS_20220818T104434.SAFE\GRANULE\L1C_T33UXS_A028462_20220818T095747\QI_DATA\MSK_DETFOO_B05.jp2.gml

WARNING: org.esa.s2tbx.dataio.s2.ortho.S2OrthoUtils: Warning: impossible to read properly mypath\S2B_MSIL1C_20220818T095549_N0400_R122_T33UXS_20220818T104434.SAFE\GRANULE\L1C_T33UXS_A028462_20220818T095747\QI_DATA\MSK_DETFOO_B06.jp2.gml

WARNING: org.esa.s2tbx.dataio.s2.ortho.S2OrthoUtils: Warning: impossible to read properly mypath\S2B_MSIL1C_20220818T095549_N0400_R122_T33UXS_20220818T104434.SAFE\GRANULE\L1C_T33UXS_A028462_20220818T095747\QI_DATA\MSK_DETFOO_B07.jp2.gml

WARNING: org.esa.s2tbx.dataio.s2.ortho.S2OrthoUtils: Warning: impossible to read properly mypath\S2B_MSIL1C_20220818T095549_N0400_R122_T33UXS_20220818T104434.SAFE\GRANULE\L1C_T33UXS_A028462_20220818T095747\QI_DATA\MSK_DETFOO_B08.jp2.gml

WARNING: org.esa.s2tbx.dataio.s2.ortho.S2OrthoUtils: Warning: impossible to read properly mypath\S2B_MSIL1C_20220818T095549_N0400_R122_T33UXS_20220818T104434.SAFE\GRANULE\L1C_T33UXS_A028462_20220818T095747\QI_DATA\MSK_DETFOO_B8A.jp2.gml

WARNING: org.esa.s2tbx.dataio.s2.ortho.S2OrthoUtils: Warning: impossible to read properly mypath\S2B_MSIL1C_20220818T095549_N0400_R122_T33UXS_20220818T104434.SAFE\GRANULE\L1C_T33UXS_A028462_20220818T095747\QI_DATA\MSK_DETFOO_B09.jp2.gml

WARNING: org.esa.s2tbx.dataio.s2.ortho.S2OrthoUtils: Warning: impossible to read properly mypath\S2B_MSIL1C_20220818T095549_N0400_R122_T33UXS_20220818T104434.SAFE\GRANULE\L1C_T33UXS_A028462_20220818T095747\QI_DATA\MSK_DETFOO_B10.jp2.gml

WARNING: org.esa.s2tbx.dataio.s2.ortho.S2OrthoUtils: Warning: impossible to read properly mypath\S2B_MSIL1C_20220818T095549_N0400_R122_T33UXS_20220818T104434.SAFE\GRANULE\L1C_T33UXS_A028462_20220818T095747\QI_DATA\MSK_DETFOO_B11.jp2.gml

WARNING: org.esa.s2tbx.dataio.s2.ortho.S2OrthoUtils: Warning: impossible to read properly mypath\S2B_MSIL1C_20220818T095549_N0400_R122_T33UXS_20220818T104434.SAFE\GRANULE\L1C_T33UXS_A028462_20220818T095747\QI_DATA\MSK_DETFOO_B12.jp2.gml

done.

Error: [NodeId: S2Resampling] Invalid S2 source product.

Can we compared the versinos of the SW? I am working on a Windows Machine with:

Could this be related to the new S2 processign baseline (0400). We have discussed the posibility here

C2RCC-MSI for S2 Not Producing Output Since Jan 14

Hello @sentinel dg I can’t comment on the impact on the C2RCC processor, I’m afraid. The intricacies of the processor are unknown to me. All I can say is that the significant changes for L1C in PB04.00 involved changes to the handling of Quality Masks (they are JP2 now, not GML) and the introduction of the Radiometric Offset. Sorry I can’t be more help. Cheers Jan

[abruescas]

I am using SNAP version 9.0, with updated plugins. I downloaded the image from ESA’s Open Access Hub, so last version.

[sentinel]

will update my version and let you know! hopefully it works

[sentinel]

@abruescas I have updated to SNAP 9.0.1 (also the S3TBX) and now I do get a result. I have though a question:

• Why do you use S2 Resampling operator instead of the standar resample? Is there any advantage or specific reasion why this should be done?

[abruescas]

S2Resampling will work better in some occasions.
For instance, look here:

Tile border bug?

Hi, I have this image (S2A_MSIL1C_20220515T081611_N0400_R121_T33HYD_20220515T102520.zip) I am trying to run C2RCC. My workflow is: Resample->Subset->Idepix->C2RCC The C2RCC results are very weird (attached image rhow_B4) [rhowB4] I am wondering why this black border appears after C2RCC. This image is the Band 4 just before I run C2RCC [B4] Any ideas?Thanks in advance Lazaros Spaias

It is up to you, in any case. If you use Resample, the subset could be done in the previous step and you will need less memory to run the graph, I guess.

[sentinel]

Thanks.

Regarding the parameters to input, how should be their values if no in-situ data is available? I am refering to Temperature, Ozone, Air Pressure?

[abruescas]

You could leave what is there by default. Or if you have data form some station near by, I guess you can try to change the values.

[sentinel]

@abruescas about the valid pixel expression in the C2RCC processor, I am getting a strange result. I am working on SNAP 9.0.2 with an image from June 2022, so with the new baseline N0400.

I am running the operator on the scene with the default valid expresion B8>0 && B8<0.1

The chla and tsm outputs are empty. However, using the same valid pixel expression on an image with the previous baseline (N0209) works fine.

What is the cause of this? I can remove the valid expression and derive the outputs over the complete scene, but this would include results over the clouds. I am interested in masking out the clouds. Maybe there is an alternative for this?

[abruescas]

Yes, I noticed that the valid pixel expression was not working in the gpt .xml I uploaded here. It is something with the expression, not the bands. Maybe @marpet can help here?

[marpet]

Hi sentinel,
I’ve just processed a subset of your scene.
This is the result:

It is this narked area:

Which area do you use for processing?
Do you use the latest SNAP and S3TBX version or are you still on SNAP 8?

In some cases, the valid pixel expression is not working. For example, if the water is very turbid or is affected by glint. Then it needs to be adapted. To find a good expression you can use the mask manager and create a new mask. Change the expression till it fits your needs.
For example, this scene:

The red area is masked water by the default C2RCC expression, the blue area is water after changing the expression to B8 > 0 && B8 < 0.2
This slight change makes a big difference. Now there is too much marked, but this acceptable. Probably the OOS flags will flag these areas.
Another option to limit the processing to the area you need is to either load or create a geometry of the water body. This geometry can be used in the valid-pixel expression.

[sentinel]

@marpet @abruescas thanks for the feedback. Before addressing this, I am facing now an issue with the previous step to C2RCC

I have download SNAP from the official website. I am running 9.0.0

I am testing a 2022 image with the new baseline 0400:

S2A_MSIL1C_20220607T100601_N0400_R022_T33UVT_20220607T134902

I running a basic pre-processing step before C2RCC, which is resample and subset

When I click run, I am facing this error:

Any clue what is going on? I have re-downloaded SNAP to start from 0 and still having this issue. It is blocking to move on to the C2RCC step.

[marpet]

Hi sentinel,

I’m sorry to hear that you run right into the next issue.
This error was reported before:

GPT Resampling Error "dataType out of range" with S2 MSIL1C data (since PB400?)

Hello all, my script to resample S2 MSIL1C data with GPT recently fails with: Error: dataType out of range! Older data and also resampling the same data with SNAP (8.0.9) is working just fine. I wonder if this is an issue with the latest Processing Baseline 04.00? I saw this post (S2 error in resampling) describing a different problem with PB400. It shouldnt be a memory issue, but I cant figure out what else it might be. The resampled products are created, but it breaks with the error half…

and

Sentinel-2 Resample and Subset - Not performing subset

Hi, I have 264 S2 images of a lake and I want to resample them to 20m resolution and then subset to the ROI. The process has been working well over the past day and I have processed around 194 images, however, I am now receiving a message saying “info: org.hsqldb.persist.logger: database closed” after the resampling process is complete and no subset is being carried out. I am running the code in R using the gpt, when I try to subset the resampled images using the desktop app it works. I will i…

It seems to occur on some products, but as far as I know the actual reason has not been figured out.
I believe to remember the using the S2Resampling operator instead of the generic resampler can overcome this problem. It is anyway a good Idea to use S2 specific resampler for S2 data.

Maybe @FlorianD has some new insights regarding this issue.

[sentinel]

@marpet Thanks for the feedback.

I am trying your idea with the S2resampling operator but when putting it on the graph I get:

Is this a known issue?

[marpet]

It has been reported in the past.
Search results for ‘invalid s2 source product’ - STEP Forum (esa.int)

In some it is said that it is due to corrupt data in other to limited compatibility of the S2Resampling operator with the Graph Builder. If tried the S2Resampling in the Graph Builder with the data from my previous example and got the same error. So, it seems it is an issue with the Graph Builder.
If you want to set up a graph you can still do and save the XML file. Then you can use it with gpt from the command line.
If you just want to execute it in SNAP Desktop you can use the standalone operators from the main menu. Execute each on the selected source product without saving to disk (except you want to keep some intermediate result). Save only the last step to disk. This is my preferred workflow as long I don’t use gpt.

@FlorianD Can you add this as an issue to our bug tracker.