You last post indicated that the attempt to build a jpy wheel that works with your python-3.6 failed.
This may have replaced /usr/local/bin/python3 with the homebrew version, but you may still have /usr/local/bin/python3.6. Check that /usr/local/bin/python3 --version gives version 3.6 and if not, try using /usr/local/bin/python3.6 in place of /usr/local/bin/python3
It isn’t enough to have OpenJDK JDK8 installed, as it is needed for the /usr/local/bin/python3.6 setup.py bdist_wheel step to build jpy for your version of Python. In a terminal, set and verify JAVA_HOME:
export JAVA_HOME=<directory containing your OpenJDK JDK-8>
$JAVA_HOME/bin/java -version
You also need to install maven.
If you aren’t sure that you have the wheel package installed for your Python-3.6:
@gnwiii Thank you very much for your help. It has been very useful. I have been finally able to install Python Snappy on my macOS.
I would like to ask a last question. Would it be possible to link Snappy to Pycharm? I know that it is possible with Jupyter, but I haven´t found any doc about it.
I would like to collaborate with the wiki, will I need to contact with @marpet? I dont´t know how to continue with the configuration article.
Kind regards
Yes, it is possible to use snappy with PyCharm.
Actually this should work more or less automatically.
In PyCharm you need to configure the Python instance you want to use. This can also be an Anaconda environment. If you select the environment you have configured for snappy you can use it.
First, I’m very glad you got the install to work. Others have encountered the same issues and don’t always stick with it as you have done. Few macOS users are aware of Apple’s “always use the full path for non-system programs” rule, so that has been a major source of confusion. You should mention the python version that you ended up using – at last report you had Python.org python-3.6 and Homebrew python-3.8.
Your next steps (as mentioned in the wiki article) should be to:
test the installed snappy to make sure it works
decide how to make ESA SNAP snappy available to your own python
scripts (and avoid conflicts with other packages called “snappy”)
You may find it useful to take a step back and spend some time on command-line basics. The wiki article assumes the reader has some experience in command-line and python programming for their platform. My experience in teaching workshops has been that it is important for users who lack this background to start with a couple sessions on the command-line. You should review the steps you took so far to be sure you understand them.
I don’t think it is a good use of resources to update the wiki article for those who have little previous command-line experience: a) such knowledge is best gained using external resources, and b) it would be harder for experienced users to locate the content they need.
It would, however, be useful to provide details of a few “baseline” configurations for cases where the initial snappy configuration fails.
I´m sorry I have had a confusion between to continue and to collaborate =)
However, I have thought that it would be useful to add specific requirements about Snappy installation to macOS users. For example, that is necessary a full version 8 Java Development kit and Apache maven.
This requirement would be obvious to an experienced Java developer, but could be mentioned (with details of versions) in a document on “what to do when snappy-conf fails”. Again, it is important to avoid overloading documents like the wiki article with details that are 1) already explained elsewhere and 2) only needed when something is broken. When something is broken, the best route is what you have done – ask on the forum. I think the wiki could be improved by adding some advice on the background needed to use SNAP snappy (e.g., command-line and python experience). The error message when snappy-conf fails is cryptic. It could help to add a link to the FAQ on how to report problems in the forum (the FAQ recommends searching before posting, as you have done).
macOS is particularly problematic because older versions were very similar to linux so both could be lumped together under 'UNIX". Apple has made changes a) to improve security, and b) to minimize problems encountered by users who have minimal command-line experience. This makes it more difficult for developers who work mainly with linux to advise macOS users. Fortunately, Apple does document their command-line environment:
The wiki and/or FAQ may need to add a link to this reference.
I’m experiencing similar problems to others in this thread trying to configure Snappy on my M1 Macbook. I have installed Snap, a recent JDK, Maven, built JPY, and copied the wheel across to the snappy library directory.
In my case Python expects a runtime library of 11_2, but my jpy wheel is 11_0 and I can’t see an obvious way to either install Python for an earlier version or build jpy for 11.2. Any suggestions would be appreciated!
Perhaps a more serious problem for me will be that there doesn’t seem to be a convenient way to install python versions below 3.8 on a mac with apple silicon. Is Snappy likely to work with more recent Python versions, or is 3.6 a hard ceiling?
Thanks, I’ll persevere and see if I can figure this out then.
Some progress, as much for the purposes of documenting my adventure as anything:
I updated my OS to 11.3, then rebuilt jpy. It still built a 11_0 wheel, but I read here that at least in some cases you can just rename the wheel to the version you want and it will still work. So I copied it into the snappy dir and renamed it to be 11_3. Then running snappy-conf I get further than before:
INFO: Installing from Java module '/Applications/snap/snap/modules/org-esa-snap-snap-python.jar'
WARNING: Architecture requirement possibly not met: Python is arm64 but JVM requires x86_64
INFO: Installing jpy...
INFO: Unzipping '/Users/rob/Library/Python/3.9/lib/python/site-packages/snappy/jpy-0.10.0.dev1-cp39-cp39-macosx_11_3_arm64.whl'
INFO: Configuring jpy...
INFO: jpy Python API configuration written to '/Users/rob/Library/Python/3.9/lib/python/site-packages/snappy/jpyconfig.py'
INFO: jpy Java API configuration written to '/Users/rob/Library/Python/3.9/lib/python/site-packages/snappy/jpyconfig.properties'
INFO: Configuring snappy...
INFO: snappy configuration written to '/Users/rob/Library/Python/3.9/lib/python/site-packages/snappy/snappy.ini'
INFO: Importing snappy for final test...
ERROR: Configuration failed with exit code 30
Not much info, but if I start python and try it myself
>>> import snappy
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/rob/Library/Python/3.9/lib/python/site-packages/snappy/__init__.py", line 64, in <module>
jpyutil.preload_jvm_dll()
File "/Users/rob/Library/Python/3.9/lib/python/site-packages/snappy/jpyutil.py", line 356, in preload_jvm_dll
return ctypes.CDLL(jvm_dll_file, mode=ctypes.RTLD_GLOBAL)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ctypes/__init__.py", line 374, in __init__
self._handle = _dlopen(self._name, mode)
OSError: dlopen(/Applications/snap/.install4j/jre.bundle/Contents/Home/lib/server/libjvm.dylib, 10): no suitable image found. Did find:
/Applications/snap/.install4j/jre.bundle/Contents/Home/lib/server/libjvm.dylib: mach-o, but wrong architecture
Looks like the prebuilt libjvm.dylib is the problem now. So I tried installing an arm64 mac jdk (from here) and copying libjvm.dylib across to replace the one in snap, but then python crashes instantly upon trying to import snappy. The OS error log gives Termination Reason: Namespace CODESIGNING, Code 0x2… which doesn’t provide many useful google results.
Slight extra progress. I realised that it wasn’t just libjvm.dylib that needed replacing, but that it was part of a whole jdk packaged within Snap (probably obvious to a Java dev, but I’m pretty clueless about this stuff).
I copied in my new jdk to replace it, which got import snappy past that hurdle, but then it hits the same “wrong architecture” error with GDAL. I could also try to build that myself and copy it in, but I suspect that fiddling too much with the contents of the Snap installation will leave me with neither Snap nor Snappy functioning, so will restore the old jdk for now.
3.6 is not a hard ceiling, but it is widely used so well-tested and easier for others to replicate any problems you might encounter. If you plan to make extensive use of other Python libraries then you may want a more recent Python version. Note that
some Python libraries need Fortran libraries not widely available for native M1.
You didn’t mention which Python 3.8 you are using. I expect you need to stick with Rosetta to run Intel versions of Python and OpenJDK for SNAP. In any case, you should avoid using Apple’s Python as that is reserved for system processing and may change with updates. There are many 3rd party package systems for Intel macOS (homebrew, macports, fink) as well as sources and binaries from the Python Software Foundation.
I think I’ve got it working. The trick I was missing was that if you run Terminal using Rosetta then macports will install everything for x86_84. If you don’t, packages such as openjdk (for maven) and python 3.6 simply fail to install because no arm64 build exists.
So overall the procedure was:
Make Terminal open using Rosetta (find app in /Applications/utils, right click -> Get info -> Open using Rosetta)
Install Python 3.6, pip 3.6 and maven using macports
Install Python 3.6 wheel package using pip
Build jpy using Python 3.6
Make dir for snappy library and copy jpy wheel into it, changing 11_0 to 11_2
Run snappy-conf (again specifying Python 3.6)
Glad to see you have made such great progress. Getting 11.0 binaries while running as 11.2 may mean you have an older Xcode version. Since renaming the binary wheel seems to work, the changes from macOS 11.0 to 11.2 may not have affected the interfaces needed by snappy. Macports, however, recommends always using the latest Xcode version, which is now 12.5 with 11.3 support and probably fixes some bug in older versions. If you encounter issues you may need to rebuild your toolchain locally with the current Xcode. In the past I have had to install two Xcode versions, the latest for use with macports and the orginal version for the current macOS for other projects. I think you need a free developer account to get interim Xcode versions.
The Xcode 12.5 Release notes describe a change in the handling of version properties:
Don’t use the iOS MinimumOSVersion information property list key to declare the minimum release of macOS in which your app runs. Use LSMinimumSystemVersion instead. (73890473)
Future releases of macOS ignore the MinimumOSVersion key in Mac apps, including apps built with Mac Catalyst.
Future releases of macOS use the LSMinimumSystemVersion key in iOS apps built with Xcode 12.5 or later. If an iOS app doesn’t include an LSMinimumSystemVersion key, future releases of macOS compare the app’s MinimumOSVersion with the version of its Mac Catalyst runtime to determine compatibility.
