DATA Conversion issue!

I have found an issue for data conversion which I am struggling with for a couple of days!
When I get the water mask through band math everything looks fine.

But I need to save that water band. That s where the problem starts. I have tried to save it preferably in geotiff or even .hdr. but surprisingly the image boundary which carried the NaN value for the mask initially created, now in other formats carry the same value given to water bodies (255 here)! All image background turned to white (like a flood zone :neutral_face:).

Any comment is truly appreciated!

isn’t this the same as here?

Thanks, but that issue was different. Now I have the bandmath without any problem. here after saving the band math, this problem happens.

did you create a virtual band in the band maths operator?

Did you try to convert it to a real raster (and saving the product) before exporting?

Yes, I convert the virtual band. But I have two issues now.
A couple of times I have tried to save the product including the input image and band extracted, but when I open the product saved in SNAP, it is absolute black!!!

Perhaps I should go first to open a new topic why Saving product does not work! :neutral_face: Then backing to the same issue here!

Thanks again

Yes, similar issue. Last time it was not fixed, and i used some other software which i dont have access to anymore. I really need to get it done in SNAP. but its odd that saving the band in other formats result sth like this which is not usable for the further analysis! I can share the data to take a look, but it is heavy to be uploaded here.

is masking the image area an option?

The final objective is to mask the water bodies. It needs to be saved in geotiff. In SNAP, after banthmath it looks fine (water in white (1), land in black (0), and image background (NaN).

After data conversion, the problem starts! No clue!

It seems the data conversion operation has an issue. It’s not handling NaN values correctly.
There is also the question how it should do it when converting the data from float32 to uint8.
uint8 has no NaN value and so NaN becomes 255, for some reason.
It would be good if the data conversion asks for the value which shall replace Nan values.
How can you over com this?
Either don’t wright NaN values with your Band Math or you use the Band Maths again and replace the NaN values manually.
The expression might look like:

nan(water_mask) ? 128 : water_mask

This will replace every NaN value with 128 and preserve the other values (0,255)

2 Likes

Marpet, Thanks a lot for your good info. I have tried nan(water_mask) ? 128 : water_mask on the water mask produced, but the value of NaN still exists! Nothing carries the value of 128. I have even checked the properties, the same as the original band!

Is there any way I can share the data with you? Maybe I am missing sth which I haven’t notice yet! I really need to get it fixed. I have done all this processing to get to the end which is water mask maps in geotif.

Thanks.

As Marpet suggested, the target no data value should come from a user input parameter.
I’ll add this to the operator.
Thanks

1 Like

The problem is that it does not accept any new value for NaN. I tried to change NaN for bandmask produced in SNAP to 128, the new band math still shows NaN, nothing changed in the properties! Any temporary solution appreciated!

I think you’ve been hit by wrong or at least confusing behaviour in the Band Maths. Also I am.
The nan() function really checks the value for nan. If it is only marked as invalid it’s not returning true. Even in the Pixel Info view we show NaN. Maybe better to show Inv there. I’ve created recently a ticket (SNAP-697) to be able to check for invalid pixels in the Band Maths.
However, you can overcome this issue.

Use this equation for your mask:

feq(Sigma0_HH_Dissimilarity, 0) ? 0 : Sigma0_HH_Dissimilarity/10<1

On the new band open the properties and remove the valid pixel expression. Now only the water pixels are one, all others are zero.
Now you can convert it and store it to GeoTIFF.

3 Likes

I am posting here what I wrote in this other thread since it seems related to what is being discussed here.
http://forum.step.esa.int/t/isolated-nan-pixels-in-coherence-image/4920

The problem I have is that SNAP seems to be converting pixels with a zero value in the I or Q component of S1 SLC images into NaN. These NaNs are then propagated through a graph run on gpt. In that particular graph, the problem results in a coherence image with NaN values. SNAP converts those NaN values back to 0, but you would expect coherence values close to 1 for those pixels.
Complex numbers with zero-valued I or Q compenents should be treated as such, not as NaN. Is there a way to force gpt to do this?

Is it maybe caused by a division by zero? This could be a reason.
I don’t know the algorithm, maybe @lveci can shed some light.

Marpet, Thank you so much for the proposed solution!

Right now the only problem is that both image background (originally NaN) and non-water (land) have the same value of 0. This will cause issues for temporal change detection since each scene has a different size, so that we need to calculate the non-water areas for each image. I am wondering whether there is any way after data conversion, the background still carries NaN, non-water 0, and water 255 (8bit)?

I do appreciate your time and assistance.

I believe a fix for this made it into the 5.0.3 snap-engine update. Please go to tools -> plugins and check for updates.

1 Like

I use SNAP 5 with updated plugins. Do you know by any chance why the batch processing does not accept the new band math algorithm, while the band math itself does not give any error!

Thanks

@SAR2016
You can change the expression to

feq(Sigma0_HH_Dissimilarity, 0) ? 128 : Sigma0_HH_Dissimilarity/10<1 

Then you can distinguish the values.

@lveci
I think the fix does not help. Because the pixels are not checked for NaN or invalid only the target no-data value is set.
The invalid pixels are still converted to 255 even if I set the target no-data value to zero.

1 Like

By changing it to 128, again the same issue will arise, the water boundary is gone…