Snappy: Where to start?

[abgbaumann]

I always get the RuntimeError: java.lang.OutOfMemoryError: Java heap space when processing S2 data. So I already changed it as you described it in Topic 1102. I changed jpyconfig.py and snappy.ini in C:\Users\Andreas Baumann\.snap\snap-python\snappy:

But it did not really work.

[bjuhls]

Hi, did you guys already make the mosiac code running?
Im failing trying to write a script making mosaics of .dim-files…
Anybody with here who could show me an example script how to use the mosaicing with snappy?!
Thanks a lot!

[abgbaumann]

Good afternoon,

I was busy with other projects, so that I did not continue with the mosaic code. Attached two files regarding Mosaicing. If you don’t get the “out-of-memory” error, then it should theoretically work.

Thank you for any feedbacks.

Cheers,
Andreas

snappy_mosaic.py (12.5 KB)
snappy_basic.py (8.5 KB)

[bjuhls]

Thanks a lot Andreas,
I will try it for me within next days.
When i made subsets with snappy I got the “Java Heap Space” Error as well if you mean that.
The allocation of more memory didnt work for me at all.
Do you have any new solutions to that?

Cheers,
Bennet

[abgbaumann]

Hello Bennet,

I meant exactly this error. The allocation of more memory also did not work for me. But I only have 8GB RAM and if you consider that two S-2 datasets have approx. a size of 10GB, then I guess there is no solution to run it on my computer.

Cheers,
Andreas

[bjuhls]

Hi Andreas,
I have even 12GB but as you mentioned its still not enough.
For subsetting numerous files I just wrote a script which is starting a python script again and again and checks which files are already subsetted and which not, since the memory error disapears after restarting python.
Thank you again for the codes you sent!
Bennet

[bjuhls]

Me again,
I tried your script Andreas, it’s working perfectly!
Since i made the subsetting before I don’t get any Memory-Errors here anymore because of the smaller file-sizes.
I included a part using MosaicOp$Condition as well and this also works fine.
Bennet

Edit: If I include too many files I of course get the HeapSpace Error again…so I split up and create several mosaics and mosaic the smaller mosaics again, hope its understandable

[jkravitz]

Hi, is there any way to access the “How to use the SNAP API from Python” page without having use an atlassian account?

Thanks,
Jeremy

[pandza]

Hey,
I am trying to force snappy to open 10 m resolution of Sentinel-2 product but I don’t succeed. As abgbaumann said
ProductIO.readProduct(file, "SENTINEL-2-MSI-10M-UTM35N") produces error RuntimeError: no matching Java method overloads found. I didn’t make it work with the solution for S3 at https://senbox.atlassian.net/browse/SNAP-343.
Can you help me out in case of S2? How to do it?

(I think that would be a solution to my problem of NDVI caluclation. I always get results in 60 m resolution. Instead of 1098010980 px, my result is always 18301830 px)
Thanks

[abgbaumann]

Hello pandza

Did you add the following code too?

HashMap = jpy.get_type('java.util.HashMap')
GPF.getDefaultInstance().getOperatorSpiRegistry().loadOperatorSpis()

Cheers,
Andreas

[pandza]

Yes, yes, I already had that piece of code in my code.

Cheers
Milos

[marpet]

How do you create the file variable?
I think it is a string, right?
That’s the reason why no java method can be found. When specifying the format, the file needs to be Java file object.
Do it like:
ProductIO.readProduct(File(file), "SENTINEL-2-MSI-10M");
I think there is now specific- UTM-format for the 10m resolution.
But doing it like this will give you only the 10m bands.

Probably the best way to prepare the data is to read in the data in multi-size resolution format and then doing the resample to 10m.

Here I show how to do it.

Handling sentinel data in python ( working with snappy)

Regarding the ArrayIndexOutOfBoundsException I can't help you. At least I would need some more information. Is more printed as just the exception. Some stack trace below. You might be right that it might be related to the multi size issue. Subsets can currently only be done on product where all bands have the same size. In the future this should also work for multi-size products. Before you Subset you should use the resample operator, to make the bands equal in size. HashMap = snappy.jpy.get…

The call to GPF.getDefaultInstance().getOperatorSpiRegistry().loadOperatorSpis(), which Andreas mentioned
is not necessary any more, since SNAP version 5.0.

[pandza]

Yes, I create file as string.
For ProductIO.readProduct(File(file), "SENTINEL-2-MSI-10M") what do I need to have File (Error message: ‘File’ is not defined) ? Is it from snappy import File or something else?
I will update SNAP and try everything and give feedback, but for now I see I’m using SNAP 3.0. I realised if I put
HashMap = jpy.get_type(‘java.util.HashMap’)
GPF.getDefaultInstance().getOperatorSpiRegistry().loadOperatorSpis()
after ProductIO.readProduct(File(file), "SENTINEL-2-MSI-10M") it works with 60 m resolution and if I put it before it works with 10 m resolution. It doesn’t mind if I stated SENTINEL-2-MSI-10M or SENTINEL-2-MSI-60M, it just doesn’t care. For 20 m resolution I would have no clue. Is it possible that it remembers something somewhere in memory? Is there any tool to clear before product.closeIO()? It’s just my guess.

One more question is that in the end I get NDVI for e.g. 10 m resolution, but no projection. Can I somehow read EPSG from my input file and write it to my output file, and if yes how?
Regards,
Milos

[marpet]

Is it from snappy import File or something else?

Yes that’s correct.
Alternatively you can do:

File = jpy.get_type('java.io.File')

One more question is that in the end I get NDVI for e.g. 10 m resolution, but no projection. Can I somehow read EPSG from my input file and write it to my output file, and if yes how?

Why don’t you have any projection? How do you create the NDVI?
You can ask a product/band for a GeoCoding and copy it to your target.
You can use ProductUtils.copyGeoCoding(source, target);

[pandza]

I just didn’t have a line for projection at all. Now I have a projection, thank you for the answer.
In addition, I create NDVI as a new empty product and then fill it with data; I use band math expression.

Last in this round of questions, when I calculate NDVI using SNAP GUI I use tool Optical/Thematic Land Processing/ NDVI Processor. Resulting product has bit depth 128, and result from my code has 32 bit depth. In my code I stated ProductData.TYPE_FLOAT32. What should be stated in order to have a bit depth 128? Is there a list of supported formats given somewhere?

[marpet]

Where do you see the bit depth of 128?

The ndvi band is of type float32. The ndvi_flags is of int32 and then there is the flag band of the source product.
Supported as band data type are
ProductData.TYPE_INT8
ProductData.TYPE_INT16
ProductData.TYPE_INT32
ProductData.TYPE_UINT8
ProductData.TYPE_UINT16
ProductData.TYPE_FLOAT32

[pandza]

I see 128 bit depth when I do right click/ Properties in My Computer on the tif file exported from SNAP GUI. That file has 919 MB size, and my tif file generated from my python code has 459 MB and bit depth 32.

[djamainajib]

Hello,
I m trying to install snappy as explained in https://github.com/andrix/python-snappy
But, when is use this code: python setup.py install
I get this error message:

help?

Thank you

[marpet]

It seems you are using a different snappy.
This snappy in this forum refers to python binding for SNAP. It is not related to the Google compression library.

[Nobody]

@Marpet: Regarding the comands you gave for information on the operators
“gpt -h Reproject” and “gpt -h”, where should I write them?
I’m using Eclipse as my python IDE, run through OSGeo. I tried the above comands both to the comand line of windows and to the comand line of OSGeo, as well as to Eclipse’s console but nobody recognises gpt.

I’m not an expert user so I would appreciate a detailed answer, if possible. Thank you.