Sen2Cor does not run in SNAP for us

Optical Toolbox sen2cor
#1

Ubuntu 14.04 LTS … SNAP 5.0, it runs OK

Sen2Cor installed under Anaconda in users’ home directory, just like instructions… when we try to use Sen2Cor, this appears in SNAP message log:

INFO [org.esa.snap]: File “/home/dstrom/anaconda2/lib/python2.7/site-packages/sen2cor-2.3.1-py2.7.egg/sen2cor/L2A_Process.py”, line 7, in
INFO [org.esa.snap]: from tables import *
SEVERE [org.esa.snap]: ImportError: No module named tables
INFO [org.esa.snap]: Process exited with value 1
INFO [org.esa.snap]: Finished tool execution in 0 seconds

So, why is L2A_Process.py looking for module tables in python 2.7? How do I fix?

This is just what is in the install instructions:
dstrom@afsisdata2:~$ python
Python 2.7.12 |Anaconda 4.2.0 (64-bit)| (default, Jul 2 2016, 17:42:40)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2

–
David

#2

can you please enter pip list in your command line and see if tables is listed?
If not you can use pip install pytables or conda install pytables.

Please also see here:

#3

Thanks for any help, here is what you’ve asked me to do: pip list and conda install pytables (at the end).

Dave

2:~$ which pip
/home/dstrom/anaconda2/bin/pip
dstrom@afsisdata2:~$ pip list
alabaster (0.7.9)
anaconda-clean (1.0)
anaconda-client (1.5.1)
anaconda-navigator (1.3.1)
appdirs (1.4.3)
argcomplete (1.0.0)
astroid (1.4.7)
astropy (1.2.1)
Babel (2.3.4)
backports-abc (0.5)
backports.shutil-get-terminal-size (1.0.0)
backports.ssl-match-hostname (3.5.0.1)
beautifulsoup4 (4.5.1)
bitarray (0.8.1)
blaze (0.10.1)
bleach (2.0.0)
bokeh (0.12.2)
boto (2.42.0)
Bottleneck (1.1.0)
cdecimal (2.3)
certifi (2017.1.23)
cffi (1.7.0)
chest (0.2.3)
click (6.6)
cloudpickle (0.2.1)
clyent (1.2.2)
colorama (0.3.7)
conda (4.3.16)
conda-build (2.0.2)
configobj (5.0.6)
configparser (3.5.0)
contextlib2 (0.5.3)
cryptography (1.5)
cycler (0.10.0)
Cython (0.24.1)
cytoolz (0.8.0)
dask (0.11.0)
datashape (0.5.2)
decorator (4.0.10)
dill (0.2.5)
docutils (0.12)
dynd (0.7.3.dev1)
entrypoints (0.2.2)
enum34 (1.1.6)
et-xmlfile (1.0.1)
fastcache (1.0.2)
filelock (2.0.6)
Flask (0.11.1)
Flask-Cors (2.1.2)
funcsigs (1.0.2)
functools32 (3.2.3-2)
futures (3.0.5)
GDAL (2.1.0)
gevent (1.1.2)
Glymur (0.8.6)
greenlet (0.4.10)
grin (1.2.1)
h5py (2.6.0)
HeapDict (1.0.0)
html5lib (0.999999999)
idna (2.1)
imagesize (0.7.1)
ipaddress (1.0.16)
ipykernel (4.5.2)
ipython (5.3.0)
ipython-genutils (0.2.0)
ipywidgets (6.0.0)
itsdangerous (0.24)
jdcal (1.2)
jedi (0.9.0)
Jinja2 (2.8)
jsonschema (2.6.0)
jupyter (1.0.0)
jupyter-client (5.0.0)
jupyter-console (5.1.0)
jupyter-core (4.3.0)
lazy-object-proxy (1.2.1)
llvmlite (0.13.0)
locket (0.2.0)
lxml (3.6.4)
MarkupSafe (0.23)
matplotlib (1.5.3)
mistune (0.7.4)
mock (2.0.0)
mpmath (0.19)
multipledispatch (0.4.8)
nb-anacondacloud (1.2.0)
nb-conda (2.0.0)
nb-conda-kernels (2.0.0)
nbconvert (5.1.1)
nbformat (4.3.0)
nbpresent (3.0.2)
networkx (1.11)
nltk (3.2.1)
nose (1.3.7)
notebook (4.4.1)
numba (0.28.1+0.gfe99fbc.dirty)
numexpr (2.6.1)
numpy (1.11.1)
odo (0.5.0)
openpyxl (2.3.2)
packaging (16.8)
pandas (0.18.1)
pandocfilters (1.4.1)
partd (0.3.6)
path.py (0.0.0)
pathlib2 (2.2.1)
patsy (0.4.1)
pbr (1.10.0)
pep8 (1.7.0)
pexpect (4.2.1)
pickleshare (0.7.4)
Pillow (3.3.1)
pip (8.1.2)
pkginfo (1.3.2)
ply (3.9)
prompt-toolkit (1.0.14)
psutil (4.3.1)
ptyprocess (0.5.1)
py (1.4.31)
pyasn1 (0.1.9)
pycairo (1.10.0)
pycosat (0.6.1)
pycparser (2.14)
pycrypto (2.6.1)
pycurl (7.43.0)
pyflakes (1.3.0)
Pygments (2.1.3)
pylint (1.5.4)
pyOpenSSL (16.2.0)
pyparsing (2.1.4)
pytest (2.9.2)
python-dateutil (2.5.3)
pytz (2016.6.1)
PyYAML (3.12)
pyzmq (16.0.2)
QtAwesome (0.3.3)
qtconsole (4.2.1)
QtPy (1.1.2)
redis (2.10.5)
requests (2.12.4)
rope (0.9.4)
ruamel-yaml (-VERSION)
scandir (1.5)
scikit-image (0.12.3)
scikit-learn (0.17.1)
scipy (0.18.1)
sen2cor (2.3.1)
setuptools (34.3.3)
simplegeneric (0.8.1)
singledispatch (3.4.0.3)
six (1.10.0)
snowballstemmer (1.2.1)
sockjs-tornado (1.0.3)
Sphinx (1.4.6)
spyder (3.0.0)
SQLAlchemy (1.0.13)
statsmodels (0.6.1)
sympy (1.0)
tables (3.3.0)
terminado (0.6)
testpath (0.3)
toolz (0.8.0)
tornado (4.4.2)
traitlets (4.3.2)
unicodecsv (0.14.1)
wcwidth (0.1.7)
webencodings (0.5)
Werkzeug (0.11.11)
wheel (0.29.0)
widgetsnbextension (2.0.0)
wrapt (1.10.6)
xlrd (1.0.0)
XlsxWriter (0.9.3)
xlwt (1.1.2)
You are using pip version 8.1.2, however version 9.0.1 is available.
You should consider upgrading via the ‘pip install --upgrade pip’ command.
dstrom@afsisdata2:~$ conda install pytables
Fetching package metadata …
Solving package specifications: .

All requested packages already installed.

packages in environment at /home/dstrom/anaconda2:

pytables 3.3.0 np111py27_0

#4

No one has any help to offer?

I provided the information requested, so I was hoping someone could help.

Any suggestions at all?

#5

If the process is still stating that tables is missing, there must be another python version on your PC.

Any other programs that use python?

#6

Yes, the Ubuntu Python packages from Ubuntu repository are installed and required for other software that we’re using on this server.

I have Anaconda set to be first in PATH when running SNAP.

dstrom@afsisdata2:~$ python -V
Python 2.7.12 :: Anaconda 4.2.0 (64-bit)

#7

so that should be alright.

Can you test pip uninstall pytables and then install it again with pip install pytables?

#8

Did you mean “conda uninstall pytables” ?

dstrom@afsisdata2:~$ pip uninstall pytables
Cannot uninstall requirement pytables, not installed
You are using pip version 8.1.2, however version 9.0.1 is available.
You should consider upgrading via the ‘pip install --upgrade pip’ command.
dstrom@afsisdata2:~$ pip install pytables
Collecting pytables
Could not find a version that satisfies the requirement pytables (from versions: )
No matching distribution found for pytables
You are using pip version 8.1.2, however version 9.0.1 is available.
You should consider upgrading via the ‘pip install --upgrade pip’ command.
dstrom@afsisdata2:~$ which pip
/home/dstrom/anaconda2/bin/pip
dstrom@afsisdata2:~$ history | grep pytable
896 pip install pytables
898 conda install pytables
959 conda install pytables
1012 pip uninstall pytables
1013 pip install pytables
1015 history | grep pytable
dstrom@afsisdata2:~$ conda uninstall pytables
Fetching package metadata …
Solving package specifications: .

Package plan for package removal in environment /home/dstrom/anaconda2:

The following packages will be REMOVED:

pytables: 3.3.0-np111py27_0

Proceed ([y]/n)? n

Exiting

#9

it says it is not installed but offers to uninstall it. I’d give it a try and proceed with y

You can use conda as well. Try to install pytables again after proper uninstalling.

You can also download it from here and use
pip install [insert corresponding whl file]
http://www.lfd.uci.edu/~gohlke/pythonlibs/#pytables

#10

As I expected, same result after conda unintstall pytables followed by conda install pytables…

dstrom@afsisdata2:~$ conda uninstall pytables
Fetching package metadata …
Solving package specifications: .

Package plan for package removal in environment /home/dstrom/anaconda2:

The following packages will be REMOVED:

pytables: 3.3.0-np111py27_0

Proceed ([y]/n)? y

dstrom@afsisdata2:~$ conda install pytables
Fetching package metadata …
Solving package specifications: .

Package plan for installation in environment /home/dstrom/anaconda2:

The following NEW packages will be INSTALLED:

pytables: 3.3.0-np111py27_0

The following packages will be UPDATED:

anaconda: 4.2.0-np111py27_0 --> custom-py27_0

Proceed ([y]/n)? y

anaconda-custo 100% |#################################################################################################| Time: 0:00:00 640.96 kB/s
dstrom@afsisdata2:~

from messages.log file:

INFO [org.esa.snap]: Traceback (most recent call last):
INFO [org.esa.snap]: File “/home/dstrom/anaconda2/lib/python2.7/site-packages/sen2cor-2.3.1-py2.7.egg/sen2cor/L2A_Process.py”, line 7, in
INFO [org.esa.snap]: from tables import *
SEVERE [org.esa.snap]: ImportError: No module named tables
INFO [org.esa.snap]: Process exited with value 1
INFO [org.esa.snap]: Finished tool execution in 0 seconds

Any suggestions?

#11

have you also tried to install it from the whl-file?

I still think that the modules you install are located in a different python version from the one sen2cor uses.

Have you had a look at the anaconda folder under \Lib\site-packages if sen2cor is in there?

#12

Doesn’t this show that sen2cor is “in there” in anaconda? I did install it in anaconda, per directions.

How can we tell which python sen2cor is using?

#13

by just typing python the first python installation found in your system is opened. It shows the installation as well.

#14

Hi David,

I was having a similar problem. If I recall correctly, the problem is because it is basically not finding python. I think all you have to do is remember to call (L2A_Bashrc is located in your SEN2COR_HOME folder)

source L2A_Bashrc

in your shell before you call L2A_Process , so that SEN2COR_HOME, SEN2COR_BIN and GDAL_DATA are exported.

Hope this solves the problem

Edit: Just realized you are calling sen2cor within SNAP, right? Maybe the env variables I mentioned above are not well defined in SNAP then?

image.png691×218 13.7 KB