Memory issues with Snappy

PederBG · June 26, 2017, 5:44pm

Hi,

I’m trying to make a simple python script that takes in a Sentinel-1 product, does some SNAP toolbox operations on it and then produce an image from the product. The snap-python interface was pretty easy to understand, but I keep getting issues with what i’m guessing is the memory usage.

java.lang.OutOfMemoryError: Java heap space
RuntimeError: java.lang.OutOfMemoryError: GC overhead limit exceeded

I also get another error when generating the image.

RuntimeError: org.esa.snap.core.gpf.OperatorException: Cannot construct DataBuffer.

I’ve read a lot of topics about this and have tried to modify the files in the snap directory, but with no luck.

The weird thing is that my program is running fine, without any errors, in Pycharm, it’s when I run it from command line it’s not working.

Are these problems only solved by increasing heap memory or can you do the SNAP operations in smaller parts/ write more efficient code?

I can post my code, stack trace and specs if that would be helpful.

I’m pretty new to this, so any help would be appreciated.

Regards Peder.

marpet · June 27, 2017, 8:06am

You say that you program runs fine when you use PyCharm but not from the command line.
This makes me think that you have two (or more) python installations.
Please check which python is invoked from PyCharm and from the command line.
Maybe you have only configured the one but not the other installation.
This is how it look for me:
In PyCharm I use Python34

and on the command line my default Python is a Anaconda 2.7 Python

PederBG · June 27, 2017, 6:45pm

Thanks for the reply. I understand your thinking, and I probably should have specified this, since it’s a common mistake. However I have made sure that both PyCharm and the command line runs in the same interpreter (python27 in my case).

This is very frustrating since I need it to run without PyCharm when I’m going to implement the script into a project later.

marpet · July 5, 2017, 8:12am

I’ve no idea why this happens. Maybe the following helps to find out what is going on.

You can enable the debug mode in the snappy. ini file. Change debug to true.

debug: True

Now you should get some log output which might help already.
If you want to get more log output you can edit the _init_.py

jpy.diag.flags = jpy.diag.F_ALL