Snappy very slow

[caleb]

Hello,

I’m trying to calculate the NDVI using the snappy_ndvi.py example buyt the processing is extremely slow, maybe more than one day to process 30978 lines. However, the time employed when I use the destock application is around 20 minuts.

What can it be happening?

Thanks in advances,
Caleb

[marpet]

Maybe the example is not well suited for a product with multiple resolutions. For a MERIS RR product with a size of 1121x14993 it just takes a few seconds.
With S2 I notice that it even starts slower and then further slows down. Probably the reader is not as fast as the one for Envisat. This is very likely caused by the data format.
After I made some changes to the script I can process almost one line per second.
When I resample the product before and store it in BEAM-DIMAP format the processing is as fast as for MERIS.

I’ve attached the updated script updated.
snappy_s2_ndvi.py (2.0 KB)

[caleb]

Thank you so much for your time.

I tried with the new version and the first 1024 lines was processed fast but after that it became slow again. Are there any possible to subset just band 4 and 8 whitout lossing resolution?, in order to test the script with a small scene.

[marpet]

Maybe it helps if you increase the memory:

Snappy error - ProductIO.writeProduct

I'm not sure, I'm not very familiar with snappy, but I think you have to set a value in the jpyconfig.py script in the snappy folder. Try setting jvm_maxmem = None to jvm_maxmem = '6G' Another place to change the memory usage might be in snappy.ini in the snappy folder. Change the line # java_max_mem: 4G to e.g. java_max_mem: 6G

You can resample the product to the 10m resolution of band B4 and B8 and afterwards you can create the subset with the two bands.

This can also be done in python.
This thread might be helpful:

Handling sentinel data in python ( working with snappy)

I have recently installed SNAP in my Windows8.1 with 8GB RAM and as I said in my first message I am trying to work with SNAP in python using Snappy. I managed to use it properly with the test dataset that is provided with the snappy module in the SNAP directory. I usually work with numpy to handle and process images or multidimensional arrays. But the problem is that the test dataset has really small size if compared with the usual sizes of a Sentinel dataset. And when I try to read for exam…

[caleb]

Really thanks for the quick replay. It is really kind of you and I am appreciating it so much.
I have another question about subset. Is it possible just subset band B4 and B8? Both bands present a 10m resolution so a resampling method is not necessary, is not?

Thank in advance.

[marpet]

You are right. Actually it is not necessary. Unfortunately our framework doesn’t allow this at the moment.
But it should be possible in the future.

[caleb]

Thank you again! I wish that option were able soon!

[GeorgeB]

Anyone got any solutions on this issue with slow computing? I’m trying to calculate a normalized index and after 1 minute it just stops but not on all Sentinel 3 files, just some of them. I set up the memory in the config files too. Anyone got better ideas?