SNAPPY performance issues

snap
1

Hello everyone,

I’ve been working on a SNAP pipeline in Python and noticed the performance was way worse than in the GUI.

By saving and reloading the product after every operation (which is counter intuitive to fast code), the pipeline went from taking 3 hours to 15 minutes.

Moreover, running the process in a thread caused problems, such as file locks not properly released in memory while the files are usable by other programs with no issues.

Cheers and thank you for your work,

ASimoneau

2

Could you please provide information about the SNAP version you are using, as well as your operating system?

3

Of course,

SNAP 11 on W11.
OpenJDK 21.0.1
Python 3.10

4

Hello,

could you please also provide a snippet of your Python code/script you are running?

Many thanks,
Olaf

5

I did a benchmark with a minimal code to get a time comparaison.
I used a smaller image just to keep the processing time reasonable.

With the standard process: 18:03
Saving and reloading: 03:41

example.py.txt (2.4 KB)

P.S.: The forum doesn’t let me upload a Python script. I had to change the extension.

Edit: I built an equivalent graph in the SNAP GUI and the processing time was about 21 minutes. So the issue seems to be more fundamental than the SNAPPY interface.

6

Any updates on this?

7

After digging around, this seems to be closely related to this post, for which there is a JIRA open with critical priority for over a year with no updates.

There is also multiple forums post discussing performance issues with GPT and SNAPPY. (Gpt performance, Performance of snap desktop and snappy, GPT batch processing abnormality, Graph Builder Editor bad performance, Multi-Threading and Performance of snappy / jpy, ProductIO.writeProduct() function takes longer to run when called in Python, Snappy running really slow)

Could we get some status update on this issue? That would be greatly appreciated.

8

Glad I’m not the only one seeing this. It’s wild that saving and reloading the product actually speeds things up, but I’ve noticed the same bottleneck with memory not being released properly in Python. It feels like the JNI/jpy bridge just struggles with the garbage collection compared to the Java GUI.