Install snappy on Ubuntu 22.04

Boorhin · July 8, 2024, 9:18am

Hi,
I am not sure my install of snap 10 is very clean and I cannot get snappy to function properly.

***@geocluster:~/Software/snap/bin$ which python
/usr/bin/python
***@geocluster:~/Software/snap/bin$ ./snappy-conf /usr/bin/python
OpenJDK 64-Bit Server VM warning: Option AggressiveOpts was deprecated in version 11.0 and will likely be removed in a future release.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.netbeans.ProxyURLStreamHandlerFactory (file:/home/***/Software/snap/platform/lib/boot.jar) to field java.net.URL.handler
WARNING: Please consider reporting this to the maintainers of org.netbeans.ProxyURLStreamHandlerFactory
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Configuring ESA SNAP-Python interface...
Configuration failed!
Error: Python configuration error
Full stack trace:
***@geocluster:~/Software/snap/bin$

It could be linked with Maven?

Hard to tell from the error message
Thanks

Corrochano · July 8, 2024, 9:37am

Hi,

I was on a similar situation and report it on this topic

I recommend you to check the logging files (default path is ~/.snap/system/var/log)

On the related post is saying that you need to upgrade Snap to 10.0.0 → 10.0.1
I can do that on the plug-in section when checking for updates.

Also, on the post that I linked, a developer told me that snappy only works properly until Python 10.

Good luck with the issue!

Regards,

Álvaro

Boorhin · July 8, 2024, 1:28pm

Thanks but this link concerns Windows 11 not Linux.
The logs indicate issues that make not much sense about Java since it is all pointing where it should…
I am going through this after a clean install of snap Desktop 10: https://senbox.atlassian.net/wiki/spaces/SNAP/pages/2499051521/Configure+Python+to+use+the+new+SNAP-Python+esa+snappy+interface+SNAP+version+10
Again nothing makes much sense as the error messages provide unusable information.
In these day and age such module should end up in ~/.local/lib/python3.10/site-packages. I just can’t figure out what to do.
Did someone test it on a Ubuntu 22.04?
Where I am so far is this:

***@geocluster:~$ python3
Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path.append('/home/***/.snap/snap-python')
>>> import esa_snappy
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/***/.snap/snap-python/esa_snappy/__init__.py", line 62, in <module>
    import jpyutil
ModuleNotFoundError: No module named 'jpyutil'

Which brings to this page

And honestly the installation instructions are a disgrace what does it even mean? 20+ years of scientific computing and I have rarely seen that in a major ecosystem.
So, I cloned the thing and exported JAVA_HOME to the system one, it crashes.

export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
python3 setup.py -v build maven bdist_wheel
jpyutil - ERROR: JAVA_HOME set by environment variable to /usr/lib/jvm/java-11-openjdk-amd64 but no no "include" or "lib" directory found.  Does not appear to be a JDK directory.

It is indeed a JDK directory… Snap works well with it.

Boorhin · July 8, 2024, 10:13pm

Hi again,
It is not at all what is recommended by the documentation but I managed to import snappy (not tested) with this method.

$ pip install jpy
Defaulting to user installation because normal site-packages is not writeable
Collecting jpy
  Downloading jpy-0.17.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (349 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 349.3/349.3 KB 1.8 MB/s eta 0:00:00
Installing collected packages: jpy
Successfully installed jpy-0.17.0
$ export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
$ export SNAP_HOME=/home/***/esa-snap
python3
Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path.append('/home/***/.snap/snap-python')
>>> import esa_snappy
WARNING: org.esa.snap.core.util.ServiceLoader: org.esa.snap.core.gpf.OperatorSpi: Provider eu.esa.opt.meris.sdr.aerosol.AerosolMergerOp$Spi not found
WARNING: org.esa.snap.core.util.ServiceLoader: org.esa.snap.core.gpf.OperatorSpi: Provider eu.esa.opt.meris.sdr.aerosol.ModisAerosolOp$Spi not found
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.esa.snap.runtime.Engine (file:/home/***/esa-snap/microwavetbx/modules/ext/eu.esa.microwavetbx.sar-cloud/org-esa-snap/snap-runtime.jar) to method java.lang.ClassLoader.initializePath(java.lang.String)
WARNING: Please consider reporting this to the maintainers of org.esa.snap.runtime.Engine
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
SLF4J: No SLF4J providers were found.
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See https://www.slf4j.org/codes.html#noProviders for further details.
SLF4J: Class path contains SLF4J bindings targeting slf4j-api versions 1.7.x or earlier.
SLF4J: Ignoring binding found at [jar:file:/home/***/esa-snap/snap/modules/ext/org.esa.snap.snap-netcdf/org-slf4j/slf4j-simple.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See https://www.slf4j.org/codes.html#ignoredBindings for an explanation.
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.esa.snap.core.util.EngineVersionCheckActivator: Please check regularly for new updates for the best SNAP experience.
>>>

It is a lot of warnings but I guess it may work outside these specific functions. Pip is clean and installs where it should. Could be good to have a pip esa-snappy? It would then fetch what is needed and install it clean and in the right place on a Linux.
Cheers,

Corrochano · July 9, 2024, 7:53am

Hi,

The thread it’s for Windows but on the posts that I linked I talk about Ubuntu because I was installing it on Ubuntu 22.04.4 LTS

Happy to see that it’s working on your computer now!

I notice with your new post that previously you are writing

which python

Instead of

which python3

So maybe the initial problem was that the path was for a previously Python version?

Anyway, I also thought that a pip install will be so good because right now the installation might be a headache.

Regards,

Álvaro

Boorhin · July 9, 2024, 8:32am

haha! That’s correct, the documentation/ installation instructions needs updating then. Totally forgot we had legacy 2.7 on ubuntu still

Thanks

dolaf · July 10, 2024, 1:23pm

Dear user Boorhin,

thanks for your reports. However, all your approaches above (manual jpy installation etc.) are no longer necessary with SNAP 10. Please refer, as you already did, only to the latest installation instructions:
https://senbox.atlassian.net/wiki/spaces/SNAP/pages/2499051521/Configure+Python+to+use+the+new+SNAP-Python+esa+snappy+interface+SNAP+version+10

We know that the jpy-consortium/jpy documentation is outdated, not very good, and even partially wrong these days. The esa_snappy installation instructions (link above) do not point you there, so please don’t use it for esa_snappy with SNAP 10.

Alvaro (thanks for still joining the discussion!) successfully installed esa_snappy on two Ubuntu 22.04 platforms, and so did I on a 20.04. Thus, it must be a specific issue in your environment. Could you provide the log file in ~/.snap/system/var/log which was generated when you tried (and failed) to configure esa_snappy? Even if it does not look useful at first glance, it may contain some hints.

Thanks both for the suggestion of providing a pip installation. This has been brought up by a couple of users, and we will consider this for a future SNAP release.

Cheers
Olaf

Boorhin · July 12, 2024, 9:56am

Hi,
Is it possible to update your instruction page as it is unclear? I strongly suggest you get it proof-read or put these in a collaborative space so it gets improved by users. You state that it would work with python 2.x but that’s what crashed the system. On other parts it is stated that it works better on 3.10 so simply suggest to do $ which python3.10 or $ which python3 to get the path to the right executable file.
For example I got this thanks to using the proper path:

snappy-conf /usr/bin/python3.10
OpenJDK 64-Bit Server VM warning: Option AggressiveOpts was deprecated in version 11.0 and will likely be removed in a future release.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.netbeans.ProxyURLStreamHandlerFactory (file:/home/***/esa-snap/platform/lib/boot.jar) to field java.net.URL.handler
WARNING: Please consider reporting this to the maintainers of org.netbeans.ProxyURLStreamHandlerFactory
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Configuring ESA SNAP-Python interface...
Configuration finished successful!
Done. The SNAP-Python interface is located in '/home/***/.snap/snap-python/esa_snappy'
When using SNAP from Python, either do: sys.path.append('/home/***/.snap/snap-python')
or copy the 'esa_snappy' module into your Python's 'site-packages' directory.
The executable of the Python environment is located at '/usr/bin/python3.10'

That’s a lot of warnings of deprecation and a typo (successful => successfully).
For testing purpose, I suggest you use something like this as an example:

import sys, os
sys.path.append(os.environ['HOME']+os.sep+'.snap'+os.sep+'snap-python')
import esa_snappy as es

or create an environment variable for that path if you don’t know how to add the package directly within the user python3.10 packages.

There are a lot of warnings about these illegal access in Java. I am afraid I am not qualified in the language but they seem pretty serious at times. The kernel dies at import at least once.
That’s from the shell where python runs

WARNING: org.esa.snap.core.util.ServiceLoader: org.esa.snap.core.gpf.OperatorSpi: Provider eu.esa.opt.meris.sdr.aerosol.AerosolMergerOp$Spi not found
WARNING: org.esa.snap.core.util.ServiceLoader: org.esa.snap.core.gpf.OperatorSpi: Provider eu.esa.opt.meris.sdr.aerosol.ModisAerosolOp$Spi not found
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.esa.snap.runtime.Engine (file:/home/***/esa-snap/microwavetbx/modules/ext/eu.esa.microwavetbx.sar-cloud/org-esa-snap/snap-runtime.jar) to method java.lang.ClassLoader.initializePath(java.lang.String)
WARNING: Please consider reporting this to the maintainers of org.esa.snap.runtime.Engine
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
SLF4J: No SLF4J providers were found.
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See https://www.slf4j.org/codes.html#noProviders for further details.
SLF4J: Class path contains SLF4J bindings targeting slf4j-api versions 1.7.x or earlier.
SLF4J: Ignoring binding found at [jar:file:/home/***/esa-snap/snap/modules/ext/org.esa.snap.snap-netcdf/org-slf4j/slf4j-simple.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See https://www.slf4j.org/codes.html#ignoredBindings for an explanation.
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.esa.snap.core.util.EngineVersionCheckActivator: Please check regularly for new updates for the best SNAP experience.
[JupyterQtConsoleApp] KernelRestarter: restarting kernel (1/5), new random ports

An that is what is printed in the jupyter qtconsole

WARNING: org.esa.snap.core.util.ServiceLoader: org.esa.snap.core.gpf.OperatorSpi: Provider eu.esa.opt.meris.sdr.aerosol.AerosolMergerOp$Spi not found

WARNING: org.esa.snap.core.util.ServiceLoader: org.esa.snap.core.gpf.OperatorSpi: Provider eu.esa.opt.meris.sdr.aerosol.ModisAerosolOp$Spi not found

WARNING: An illegal reflective access operation has occurred

WARNING: Illegal reflective access by org.esa.snap.runtime.Engine (file:/home/***/esa-snap/microwavetbx/modules/ext/eu.esa.microwavetbx.sar-cloud/org-esa-snap/snap-runtime.jar) to method java.lang.ClassLoader.initializePath(java.lang.String)

WARNING: Please consider reporting this to the maintainers of org.esa.snap.runtime.Engine

WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations

WARNING: All illegal access operations will be denied in a future release

SLF4J: No SLF4J providers were found.

SLF4J: Defaulting to no-operation (NOP) logger implementation

SLF4J: See https://www.slf4j.org/codes.html#noProviders for further details.

SLF4J: Class path contains SLF4J bindings targeting slf4j-api versions 1.7.x or earlier.

SLF4J: Ignoring binding found at [jar:file:/home/***/esa-snap/snap/modules/ext/org.esa.snap.snap-netcdf/org-slf4j/slf4j-simple.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: See https://www.slf4j.org/codes.html#ignoredBindings for an explanation.

INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters

INFO: org.esa.snap.core.util.EngineVersionCheckActivator: Please check regularly for new updates for the best SNAP experience.

Here the logs but it seems they mix with the ones of running snap desktop
messages_anon.zip (5.4 KB)

nw_young · November 2, 2024, 6:23am

I am new to the Forum, but a long term user of SNAP.

I have been getting the messages regarding “Illegal reflective access”.

Searching Java related forums provides a couple of useful posts. The problem is solely related to Java (JDK various versions). See:
reddit
/r/java/comments/p9ymhb/what_exactly_is_the_reason_for_denying_reflection/?rdt=62860

and from pingidentity
/s/article/an-illegal-reflective-access-operation-has-occurred-when-using-java-11-with-forgerock-products

This informative extract:

ROOT CAUSE OF WARNING:

These warnings are JDK enforcement warnings and are therefore external to ForgeRock products. In Java 9 through to Java 15, the –illegal-access property is set to permit by default. This setting alerts you to code on the class path that has reflectively accessed a JDK-internal API at least once, but the presence of this warning does not affect functionality. The component in question is detailed in the Please consider reporting this to the maintainers of part of the warning, and it is up to the maintainer of that component to stop using JDK-internal APIs."

I believe there is also a solution to this warning issue by changing the design of the Java code under SNAP.