Sen2Cor installation on MacOS

Hi everyone,

I am new to SNAP and I encountered error when I try to install Sen2Cor. The MacOS version I am using is 11.4 (Big Sur)

It first start with an error message below

Bundle installation failed. 
Please see the application log for details.

When I click to the application log, it shows

A java.nio.channels.ClosedByInterruptException exception has occurred.
	Click Show Details or see the messages.log file located in your /Users/<intentionally omitted>/.snap/system/var/log folder.

I tried to perform a clean installation https://forum.step.esa.int/t/sen2cor-detailled-instructions/8659 but the problem persists (at 4th step)

Grateful if anyone can help. Thank you!

Can you attach the out putof “Show Details” or the relevant portion of the referenced messages.log file?

The clean installation instructions are for SNAP 6. What version of SNAP are you using?

Thanks gnwill,

The output of “Show details”:

java.nio.channels.ClosedByInterruptException
	at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
	at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:164)
	at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:65)
	at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:109)
	at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
	at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
	at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
	at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
	at java.io.InputStreamReader.read(InputStreamReader.java:184)
	at java.io.BufferedReader.fill(BufferedReader.java:161)
	at java.io.BufferedReader.readLine(BufferedReader.java:324)
	at java.io.BufferedReader.readLine(BufferedReader.java:389)
	at java.nio.file.Files.readAllLines(Files.java:3205)
	at java.nio.file.Files.readAllLines(Files.java:3242)
[catch] at org.esa.snap.runtime.EngineConfig.loadOtherClusterNames(EngineConfig.java:248)
	at org.esa.snap.runtime.EngineConfig.load(EngineConfig.java:226)
	at org.esa.snap.runtime.EngineConfig.load(EngineConfig.java:54)
	at org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO.getPreferences(ToolAdapterIO.java:590)
	at org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO.getVariableValue(ToolAdapterIO.java:154)
	at org.esa.snap.core.gpf.descriptor.SystemVariable.resolve(SystemVariable.java:128)
	at org.esa.snap.core.gpf.descriptor.SystemDependentVariable.getValue(SystemDependentVariable.java:58)
	at org.esa.snap.core.gpf.descriptor.VariableResolver.lambda$recursiveResolve$0(VariableResolver.java:57)
	at java.util.stream.Collectors.lambda$toMap$58(Collectors.java:1321)
	at java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
	at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566)
	at org.esa.snap.core.gpf.descriptor.VariableResolver.recursiveResolve(VariableResolver.java:55)
	at org.esa.snap.core.gpf.descriptor.VariableResolver$SimpleVariableResolver.resolve(VariableResolver.java:95)
	at org.esa.snap.core.gpf.descriptor.ToolAdapterOperatorDescriptor.resolveVariables(ToolAdapterOperatorDescriptor.java:493)
	at org.esa.snap.core.gpf.descriptor.dependency.Bundle.isInstalled(Bundle.java:223)
	at org.esa.snap.ui.tooladapter.dialogs.ToolAdapterTabbedEditorDialog.lambda$null$10(ToolAdapterTabbedEditorDialog.java:458)
	at org.esa.snap.core.gpf.descriptor.dependency.BundleInstaller.installFinished(BundleInstaller.java:420)
	at org.esa.snap.core.gpf.descriptor.dependency.BundleInstaller.access$000(BundleInstaller.java:61)
	at org.esa.snap.core.gpf.descriptor.dependency.BundleInstaller$Action.call(BundleInstaller.java:462)
	at org.esa.snap.core.gpf.descriptor.dependency.BundleInstaller$Action.call(BundleInstaller.java:445)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Please see the message.log file attached. There is message.log.2 file but I cannot upload it right now.

messages.log.1 (38.6 KB)
messages.log (81.2 KB)

I tried to read the log file and it indicated that the path does not exist for for ‘…/.snap/auxdata/Sen2Cor-02.05.05-Darwin64/bin/L2A_Process’, Should I create the folder manually or it is supposed to be created automatically?

The SNAP version I am using is 8.0 and the Sen2Cor I attempted to install is Sen2Cor 255.

Thank you.

The messae.log.2 file is in here

messages.log.2 (126.7 KB)

The log files have:

SEVERE [org.esa.snap]: Failed to load clusters file from '/Applications/snap/etc/snap.cluste

This could be a problem with file permissons or attributes (e.g., SNAP installed by a different user on a shared machine, Apple security policy “protecting” you from the internet, etc.) or an installation glitch that failed to create the file. In this context, “cluster” denotes a group of SNAP modules. snap.clusters seems to the list of installed module clusters. In a terminal, what does ls -l@ /Applications/snap/etc/snap.clusters show? The @ will show extended attributes used by Apple’s security policies.

Thanks gnwiii again.

I am the only user of the machine while I have checked that I have the admin right.

The ls -l result from `/Applications/snap/etc/snap.clusters’ are as below.

-rw-r--r--  1 fe  admin    32 Aug  1 17:30 snap.clusters

I tried to change the file permission by chmod 777 and perform the installation again while same error message prompted

java.nio.channels.ClosedByInterruptException
	at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
	at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:164)
	at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:65)
	at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:109)
	at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
	at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
	at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
	at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
	at java.io.InputStreamReader.read(InputStreamReader.java:184)
	at java.io.BufferedReader.fill(BufferedReader.java:161)
	at java.io.BufferedReader.readLine(BufferedReader.java:324)
	at java.io.BufferedReader.readLine(BufferedReader.java:389)
	at java.nio.file.Files.readAllLines(Files.java:3205)
	at java.nio.file.Files.readAllLines(Files.java:3242)
[catch] at org.esa.snap.runtime.EngineConfig.loadOtherClusterNames(EngineConfig.java:248)
	at org.esa.snap.runtime.EngineConfig.load(EngineConfig.java:226)
	at org.esa.snap.runtime.EngineConfig.load(EngineConfig.java:54)
	at org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO.getPreferences(ToolAdapterIO.java:590)
	at org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO.getVariableValue(ToolAdapterIO.java:154)
	at org.esa.snap.core.gpf.descriptor.SystemVariable.resolve(SystemVariable.java:128)
	at org.esa.snap.core.gpf.descriptor.SystemDependentVariable.getValue(SystemDependentVariable.java:58)
	at org.esa.snap.core.gpf.descriptor.VariableResolver.lambda$recursiveResolve$0(VariableResolver.java:57)
	at java.util.stream.Collectors.lambda$toMap$58(Collectors.java:1321)
	at java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
	at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566)
	at org.esa.snap.core.gpf.descriptor.VariableResolver.recursiveResolve(VariableResolver.java:55)
	at org.esa.snap.core.gpf.descriptor.VariableResolver$SimpleVariableResolver.resolve(VariableResolver.java:95)
	at org.esa.snap.core.gpf.descriptor.ToolAdapterOperatorDescriptor.resolveVariables(ToolAdapterOperatorDescriptor.java:493)
	at org.esa.snap.core.gpf.descriptor.dependency.Bundle.isInstalled(Bundle.java:223)
	at org.esa.snap.ui.tooladapter.dialogs.ToolAdapterTabbedEditorDialog.lambda$null$10(ToolAdapterTabbedEditorDialog.java:458)
	at org.esa.snap.core.gpf.descriptor.dependency.BundleInstaller.installFinished(BundleInstaller.java:420)
	at org.esa.snap.core.gpf.descriptor.dependency.BundleInstaller.access$000(BundleInstaller.java:61)
	at org.esa.snap.core.gpf.descriptor.dependency.BundleInstaller$Action.call(BundleInstaller.java:462)
	at org.esa.snap.core.gpf.descriptor.dependency.BundleInstaller$Action.call(BundleInstaller.java:445)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Thank you.

The original permissions look correct, so you should set the permissions back and check extended attributes, using ls -l -@, where @ is the commercial “at” symbol, or xttr.
I’m guessing that the <snap_install_dir>/etc/snap.clusters` file should looke like:

etc
ide
platform
bin
snap
s1tbx
s2tbx
s3tbx

where the lines following snap depend on which toolboxes are installed.

Using both ls -l -@ or xattr command shows that there is no extended attributes for snap.clusters

The ls -l -@ result after changing back the permission

-rw-r--r--  1 fe  admin  32 Aug  1 17:30 snap.clusters

Your guess is right while mine one does not have s1tbx and s3tbx (which I didn’t install)

etc
ide
platform
bin
snap
s2tbx

Interestingly, after installing s1tbx and s3tbx as experiment, the snap.clusters file doesn’t change.

If something is blocking access to snap.clusters then it won’t change. Are you running snap as user “fe” or using “sudo”? Did you install SNAP using “sudo”?

I am running snap as user “fe”. The SNAP was installed by “fe” as well.

I tried to enable the root user to install Sen2Cor 255 (i.e. SNAP installed by “fe” and running by root) as well but same error occurs.

Thank you.

Using “sudo” often has side effects, either some files owned by “root” in a user’s filesystem or files placed in a location not accessible to a regular user. There have been other applications that required changes to install on Big Sur. At my lab, with most users on macOS, we usually installed SNAP in the user’s ~/Applications/snap. You could try creating a new user and installing to a location in that new user’s home directory. It should not interfere with your existing installation.

Thanks gnwiii,

I tried to install another SNAP in a new created user directory but I cannot open the new installed SNAP under bin.

So I uninstall all existing SNAP and install the SNAP in the new user’s home directory but the sen2Cor installation still failed.

messages1.log (66.5 KB)
messages.log (81.8 KB)

I’m not sure what you mean. Were you logged in as the new user (tmpsnap)? I gather that you couldn’t run /Users/tmpsnap/snap/bin/snap. Did SNAP run at the end of the install? Were you still logged in as tmpsnap when trying to run /Users/tmpsnap/snap/bin/snap. Did you try this in a Terminal session or by clicking on snap in a GUI? What does ls -l -@ /Users/tmpsnap/snap/bin/snap give?

messages1.log and messages.log seem identical when downloaded here. Both start at the same time: >Log Session: Monday, August 9, 2021 10:54:27 PM HKT, both have 824 lines, and both have:

SEVERE [org.esa.snap]: Failed to load clusters file from '/Users/tmpsnap/snap/etc/snap.clusters'
java.nio.channels.ClosedByInterruptException

Looks like some conflict with Big Sur’s security policies or possibly anti-virus software. Can you run Sen2Cor in a linux virtual machine?

Thanks gnwill,

I first tried to create a new user (tmpsnap), and then installed another SNAP in other location (i.e. /Users/tmpsnap/snap) while the SNAP installed by user fe remains. The outcome is I cannot run the SNAP installed by tmpsnap by clicking on GUI.

I then uninstalled all the SNAP (whether installed by ‘fe’ or ‘tmpsnap’), after that reinstall the SNAP by tmpsnap in ‘/Users/tmpsnap/snap’. (at this moment only one snap installed), it can be opened while installation of sen2cor failed as indicated by the log in previous reply.

I also tried to install the SNAP in an old mac machine running El Captian and seems fine (at least it prompted installation success for sen2cor after few attempt), I may try to install the SNAP in a linux VM under Big Sur’s to see if the problem resolved. The problem seems related to Big Sur’s security policy or the new M1 processor as there is no anti-virus installed for the Big Sur’s machine.

Thanks for your help once again :slight_smile:

Hi everyone,

The problem causing bundle installation failed is the Mac (Big sur) blocking the download of the installation file. This can be fixed via download the stand alone installer and install it .

The installer is available at Sen2Cor v2.5.5 – STEP

In the previous setting (inside Bundled binaries option), the location was set to REMOTE and the process button is Download and Install now. I then change the location from Remote to LOCAL and then browse the installation file from local drive and click Install and it finally success!

Hope this can help others who running SNAP on Big Sur and the developer team could fix the issue in the next release, cheers.