Handling sentinel data in python ( working with snappy)

development python
#41

I resampled one S2 product, but I don’t see resampled images in my …\AppData\Local\Temp\… directory.

I know that data is cached by the S2 reader in the directory <USER_HOME>.snap\var\cache\s2tbx\l1c-reader. This could fill up your disk and the handling of this cache needs to be improved. There are already issues for this in our bug tracker (SIITBX-277, SIITBX-288).
In SNAP Desktop it is possible to change the interval how often the cache is deleted (Tools / Options / S2TBX). But this will probably not help you.
This cached data can be cleaned by a function call. I guess the following will work.

from snappy import jpy
S2CacheUtils = jpy.get_type('org.esa.s2tbx.dataio.cache.S2CacheUtils')

S2CacheUtils.deleteCache()

If you call deleteCache() after every processed product you should get rid of this problem. If the reader-cache is the problem and not something else. Probably it would be good if you first call on the source product dispose().

1 Like
#42

I’ll make a try in the following days and I will come back with an answer :slight_smile:

#43

Hi Marco,

After a few days short break I’ve come back and start dealing again with the issue.

What I figure out was that by reading the *.zip file corresponding to the S2 imagery, the readProduct function was creating data in the folder …\AppData\Local\Temp\snap-user and that data was not removed. The dispose() method called on the product obtained by using readProduct did the trick and now the hard drive is free of unwanted data. The code is looking something like this:

…
S2CacheUtils = jpy.get_type(‘org.esa.s2tbx.dataio.cache.S2CacheUtils’)
…

product = ProductIO.readProduct(file_path)
- create product subset
- do something with product subset
product.dispose()
S2CacheUtils.deleteCache()

In this way it is possible to remove the cache from subset-ing operation and remove the unnecessary data obtained after reading the product.

I would like to know if there is a difference in this case if I use product.closeIO() instead of product.dispose().

Thanks again for the tip!

George

1 Like
#44

Good to hear that it now works for you.

The method dispose() calls closeIO().

closeIO closes only the input/output stream and dispose releases all resources which have been allocated.

In the end the implementations of both methods often do the same. I would suggest calling always the dispose method.

1 Like
#45

snappy give me none output either i give zip file or unzip file. checked in two sentinel images. Instead when i open the manifest.safe in snap software it show result…

image
image.png475×514 6.22 KB

#46

This happens either when no reader is found or when the file name is invalid. It is weird when you can open the file in SNAP Desktop, though. Can you tell me the value of ‘path1’?

#47

Here is :
‘D:\SENTINEL\S1A_IW_GRDH_1SDV_20181008T131251_20181008T131316_024045_02A0A4_4743\manifest.safe’

#48

Perhaps could it be a typo in the path? Normally it should be
‘D:\SENTINEL\S1A_IW_GRDH_1SDV_20181008T131251_20181008T131316_024045_02A0A4_4743.SAFE\manifest.safe’
When you unzip it, the product folder should be [yourProductName].SAFE and if you are opening it compressed, then the path should be:
‘D:\SENTINEL\S1A_IW_GRDH_1SDV_20181008T131251_20181008T131316_024045_02A0A4_4743.zip’

#49

try it too. but same problem. give path with distinct ways :confused:

#50

Pity, the missing ‘.SAFE’ really looked like the problem. Anyway, if the .SAFE is missing, it won’t open.

Do you maybe have multiple installations of SNAP on your computer?

Can you open other S1 products with snappy? Or any other products for that matter?

#51

yes i opened .geotif with snappy ,no error detected while reading geotif

#52

Try this:

reader = ProductIO.getProductReader(‘SENTINEL-1’)
product = reader.readProductNodes(path1, None)

Is the reader None or is it an object?

#53

You should use the path with double back-slash like ‘\\’, e.g. ‘x:\\your_dir\\imag_name.zip’

To correctly read the product you must have something like:

from snappy import ProductIO

product = ProductIO.readProduct( ‘x:\\your_dir\\imag_name.zip’)
band_names = product.getBandNames()
print (list(band_names))

If you have the correct python setup for snappy (python 3.4 !!! + snappy) then the above scratch should work.

#54

@marpet @ABraun
Hie,
In the example provided snappy_flh.py I am getting an error:
w1 = b1.getSpectralWavelength()
AttributeError: ‘NoneType’ object has no attribute ‘getSpectralWavelength’
How can it be corrected?
Thanks

#55

Hi @marpet and all, Can you please help me how I can subset or process Sentinel-1 SLC data line by line?

I am trying to apply oribit file but facing memory error. Is it possible to work on a small subset of SLC data to see if code is working or not?

#56

You can apply TOPS Split to reduce the number of bursts.

#57

Hi @ABraun thanks for your reply. Can you please correct me am I on the right track to preprocess Sentinel-1 SLC data to reach the goal of change detection.

  1. add orbit file
  2. remove border noise
  3. remove thermal noise
  4. Calibrate image to sigma0 and dB
  5. TOPSAR Deburst
  6. multilook
  7. subset to study area
  8. terrain correction
  9. radiometric correction
  10. speckle noise filter
  11. geocode/reprojection.

Please recommend what steps I have to follow. I tried TOPSAR-Split using SNAP instead of Python, but its also creating very big file as output.

#58

[quote=“shahbazbaig, post:57, topic:2222”]I tried TOPSAR-Split using SNAP instead of Python, but its also creating very big file as output.
[/quote]

have you reduced the number of bursts as in the screenshot here? Apply orbit file error in S1-TOPS Coregistration

Some remarks on the steps:

  • Border Noise removal is not required for SLC products
  • I would apply a speckle filter earlier (before the conversion to db and multi-looking)
  • Multi-looking is not mandatory. After range doppler terrain correction your pixels are converted to squares anyhow.
  • The conversion to dB only makes sense if you search for changes in the low backscatter areas. The information is kind of the same but low values get streched while high values are shifted towards the mean. Your histogram then shows a more normal distribution.
1 Like
#59

Great. Let me try this and will return to you with my results.
Thanks

#60

Hi, As per your recommendation, after trying TOPSAR-Split function, I have run all the required processes successfully.

One thing I would like to confirm here. As you said border noise removal is not necessary. For testing purpose, I tried to remove border noise but function was not detecting the selected bursts product as correct one. is it because of applying TOPSAR-split function and selecting few bursts or reason is something else?