Sentinel 3 IDEPIX mask in snappy

development python
#1

Hi. I try to create a IDEPIX mask for clouds and add it to my product. After I done it i try to write the values in a new numpy array but i get the following error AttributeError: ‘org.esa.snap.core.datamodel.ProductNode’ object has no attribute 'readPixels’
despite that my mask is datamodel.Mask not datamodel.ProductNode as seen in the image.
Anyone can help my? i lost my head trying to understand those problems in masking with snappy.
Cheers!

test
test1030×477 62.3 KB

IdePix.olci hangs
#2

Try this post:

2 Likes
#3

@MarkWilliamMatthews I got a problem here. I run IDEPIX for pixel identification, especially clouds. After I run IDEPIX, I get all the masks with getMaskGroup(). After that I get each mask individually and i cast it with jpy.cast to be a mask. When I try to write the data in an empty array, here is the problem, it takes too long. I let the PC running over night and nothing happened, the cell in jupyter notebook was still running after 12h. I run it again this morning and the RAM consumption increased to 95%, from 500mb usage on python process to around 12 gb of RAM usage in 30 minutes. After that I killed it. What can it be??

The code
NORI=mask.Group.get('IDEPIX_CLOUD')
NORI2=mask.Group.get('IDEPIX_CLOUD_SURE')
NORI=mask.Group.get('IDEPIX_CLOUD_AMBIGOUS')
my_real_NORI=jpy.cas(NORI, Mask)
nori_data=np.zeros(width*height, dtype=np.float32
my_real_NORI.readPixels(0,0, height, width, nori_data)

And here bams it’s just consumes GB of RAM.

Any idea??

#4

Hey @GeorgeB

Different approach, but maybe try this:

from numpy import zeros, where, float32, nan

# Function to return the bit number from a value
def get_bit(value, bit_number):
    """
    Return value from a bit number
    """
    return (value & (1 << bit_number)) != 0

# Read in the product
p = ProductIO.readProduct(idepix.dim)

# Get product dimensions
h = int(p.getSceneRasterHeight())
w = int(p.getSceneRasterWidth())

# Get the flags band (this is bits)
idepix_flags = p.getBand('pixel_classif_flags').readPixels(0, 0, w, h, zeros((h, w), float32)).astype('uint32')

# Read the cloud mask 
cloud_mask = where(get_bit(idepix_flags, 1), 1, nan)

This will give you the cloud mask (1=true, nan=false).

You can see the bit number for the flags by opening the idepix .dim product in SNAP Desktop and looking at Flag Codings/quality_flags in the Product Explorer.

It should take seconds, depending on the size of the file. If your file is large, subset it first for testing.

If it does not work: change your RAM settings or your code if you are iterating over many files (don’t think this is the case).

Hope it helps!

#5

Cheers man! I will try it out and let you now. :grin: :grin: