Reading OLCI L1 with snappy

Dear all,

I have used snappy (with SNAP v3.0) to read OLCI level 1 data, and I am experiencing a few issues:

  • The geometry (sun and obs relative and zenith angles) is not read correctly in SNAP. The content of tie_geometries.nc looks fine though. But the content of OZA, SZA, etc is completely different and not realistic. One example orbit where I encounter this problem is: S3A_OL_1_ERR____20160404T021431_20160404T025829_20160410T195130_2637_002_317______LN1_O_NT_001.SEN3

  • I would like to use the function readBitmask, but didn’t find how to use it.

  • Performance: file access is very slow. It takes 30s to read a full MERIS RR orbit with pyepr, and it takes almost 3 minutes to read a full OLCI RR orbit (of similar size), including 15" just to perform snappy.ProductIO.readProduct. Have I done something wrong, is it a core limitation of the snappy interface, or is there possibly room for improvement?

Many thanks for your help,
François

Hi François,

the product you mention is one of the corrupted. You can notice this when on the world map only a line is shown.
It is possible to fix this.
Open the xfdumanifest.xml file in an editor and change:

<olci:rowsPerTiePoint>14984<olci:rowsPerTiePoint>

to

<olci:rowsPerTiePoint>1</olci:rowsPerTiePoint>

Afterwards the angles should look better.
We have noticed this problem in several products and it is already reported.

The performance is currently not the best. Especially the opening. It is on us to improve here.

I would like to discourage you to use the readBitmask. The replacement depends a bit on what you want to do, but I think the following will help you:

// create a mask like this
validPixelMask = Mask.BandMathsType.create("__temp_mask", null,
                                           sourceProduct.getSceneRasterWidth(),
                                           sourceProduct.getSceneRasterHeight(),
                                           maskExpression,
                                          Color.GREEN, 0.0)
validPixelMask.setOwner(sourceProduct)
// and use it like
isValid = validPixelMask.isPixelValid(x, y)

Hi Marco,

Many thanks for your quick answer.

Concerning the geometry, I have tried to set <olci:rowsPerTiePoint>1</olci:rowsPerTiePoint>
After that, SZA and SAA look fine, but OZA and OAA still have a problem and don’t look similar to the content of tie_geometries.nc

About the bitmask: I would like to read level 1 masks, for example LAND, as a numpy array. You code snippet is very useful, could you just explain how to convert the validPixelMask to a numpy array?

Instead of calling isPixelValid you can call:

valid_mask = validPixelMask.readValidMask(0,0, 10,10, new boolean[10*10]);
np_vm = np.array(valid_mask, dtype=np.bool_)

I’m not sure about the exact code. It must be similar like this. But I’m not a Python expert. If it is not correct, maybe someone can correct.

Regarding the OZA and OAA, can you say what do you think is wrong. For me it looks reasonable.
For example when looking at the data with HDFView it looks different because it is not interpolated.
Or do you have something else in mind?

Sorry, I have tried your code but am struggling at the beginning. the snappy module has an object Mask, but Mask has no attribute BandMathsType.

About the OZA and OAA, please see the following “SNAPshot” :slight_smile: (using SNAPv3.0)

[1] is the OLCI L1 xml product modified with <olci:rowsPerTiePoint>1</olci:rowsPerTiePoint>(width is 1217 px)
[2] is the netCDF tie_geometries (width is 77 px)
warning: [1] and [2] are not shown with same zoom level.
The displayed color scale is the common color scale of OZA [1] and [2].
The color scale of OAA [1] and [2] is not shown, but goes for both from -180 to 180.
OZA should be close to zero under satellite track as on [2], but minimum OZA is about 45° on [1]
OAA should change sign under satellite track as on [2], but does not on [1].
Do you see the same thing as I do ?

Thanks, for the explanation.
Now I can see what you mean. It really looks like as we do not interpolate correctly.
We will further investigate this. The issue for this is SIIITBX-110.

Regarding the python code. I get back to you tomorrow.

What is missing in the code is to declare the java types.
add the following to you code. Just below the imports.

Color = snappy.jpy.get_type('java.awt.Color')
BandMathsType = snappy.jpy.get_type('org.esa.snap.core.datamodel.Mask$BandMathsType')

Thanks Marco !
Unfortunately I still can’t get it to work, readValidMask always return an array full of ones (see test_mask.py (819 Bytes))

The code seems to be fine. At least I can’t see any problem. Have you tried different expressions?

Regarding the angels. I can see this problem only in the OLCI ERR files. I had a look at EFR and LFR and did not see such a problem. Also for LRR, which is also an orbit file I didn’t saw it.
Maybe the problem is specific to ERR. I will further investigate this.

Now I know. readvalidMask is the wrong method. You need to use readPixels(0, 0, height, width, valid_mask). But here the data type needs to be int.

Another update:
Not only the rowsPerTiePoint is wrong but also columnsPerTiePoint. columnsPerTiePoint must be set to 16.

It seems that many, if not all, RR products have this issue.

Thanks Marco!
Setting columnsPerTiePoint. columnsPerTiePoint to 16 does the trick.
Also using the readPixels method I could read the mask. I attach my code for the record.
test_mask.py (874 Bytes)