Within the example files, for example > snappy_subset.py it seems that files are read/write, but there are no code to close them, destroy file pointer or do anything else.
Is it correct?
Should I close each file, for example by calling: product.closeIO() ?
I’m asking, because I receive strange exceptions when I try to use my functions: getSigma and getSubset together (in one Python session)
When I use only one of them, close Python and use them again, I have no issues.
My functions:
Hi,
ProductIO.writeProduct internally calls to close th product writer, so there should not be a need to call another method. Can you tell us what the exception is that you ran into?
@TonioF Thank you for response, let me provide full steps to reproduce:
Steps to reproduce
I use ‘S1A_IW_GRDH_1SDV_20160512T161044_20160512T161109_011228_010FA8_C584.zip’ Sentinel-1 file for testing
First of all I import my functions:
git clone git@github.com:kedziorm/mySNAPscripts.git
cd mySNAPscripts/
python
from myScripts import *
Now, I try to execute getSigma on the mentioned file:
getSigma(SentinelFile)
which succed:
getSigma(SentinelFile)
Starting writing to the file: ~/Testy/calibrated_S1A_IW_GRDH_1SDV_20160512T161044_20160512T161.dim
‘~/Testy/calibrated_S1A_IW_GRDH_1SDV_20160512T161044_20160512T161.dim’
Next, I try to execute getSubset on the same file:
getSubset(SentinelFile)
and there is error:
getSubset(SentinelFile)
Starting writing to the file: ~/Testy/subset_S1A_IW_GRDH_1SDV_20160512T161044_20160512T161.dim
Traceback (most recent call last):
File “”, line 1, in
File “myScripts.py”, line 142, in getSubset
ProductIO.writeProduct(sub_product, newFile, OutputType[1])
RuntimeError: java.lang.IllegalArgumentException: Empty region!
BUT, I do not receive this error if I quit the python and try to run this function again
quit()
rm -rf ~/Testy/subset*
python
from myScripts import *
This succeed without errors:
python Python 2.7.12 (default, Jul 1 2016, 15:12:24) [GCC 5.4.0 20160609] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> from myScripts import * Current _JAVA_OPTIONS: 'Not Set will be changed to '-Xmx4096m' to avoid OutOfMemoryError Picked up _JAVA_OPTIONS: -Xmx4096m >>> getSubset(SentinelFile) Starting writing to the file: ~/Testy/subset_S1A_IW_GRDH_1SDV_20160512T161044_20160512T161.dim '~/Testy/subset_S1A_IW_GRDH_1SDV_20160512T161044_20160512T161.dim'
I found two solutions for your problem: One would be to raise the memory. I was able to run both functions consecutively with ‘-Xmx6G’ . The other solution (which I prefer) though is to read the source product only once and pass it as parameter to both functions instead of the path. It’s location can then be requested with ‘product.getFileLocation().getAbsolutePath()’ when needed.
@TonioF I totally agree.
However - if I understand you correctly, the problem exists, because even if I write product file, there’s still file content stored in the memory. Each time when I open product, some part of memory is used, even if I open product within separate function and write output product.
Is there any way to Free/release my memory instead?
Moreover, ideally I would like to re-write my functions so that it will be possible to call them as follows:
getSigma(getSubset(SentinelFile))
but I am not sure - is it possible to write functions which returns product (instead of path to new file), so that I will be able to use value returned from one function as an input to another function (in my case - subset will be taken and then I will calculate the sigma)/
it should be no problem to write a function which returns a product. For instance in this example a function is written which returns a band. Not a big difference compared to a Product from a API point of view.
You can call product.dispose(). This will release all for this product allocated resource. The product will not be usable aftwards.
To release memory explicitly you can call the Java Garbage Collection.
System = jpy.get_type('java.lang.System')
System.gc()@marpet Sorry for the lack of response.
I amended my script to call dispose method. Later, I would like to amend my functions, so that they will return product not file path.
Unfortunately, I still receive an error message when calling getSigma and as I check, this error occurs in SNAP Desktop as well. I created separate thread for that: http://forum.step.esa.int/t/java-lang-nullpointerexception-when-using-calibration-operator-on-beam-dimap-file/3099