ICOR not running "GDAL "

[suthagar]

Hello
good morning , previously I did icor process in windows 10 but now i am using Ubuntu
I installed SNAP, and downloaded icor setup and installed plugin properly even after opened icor- sentinel-2 processor also i changed directory and try to run but execution completed in 5 sec

The error showing like this
python3 /opt/vito/icor/src/icor.py --keep_intermediate false --cloud_average_threshold 0.23 --cloud_low_band B02 --cloud_low_threshold 0.2 --aot true --aerosol_type RURAL --aot_window_size 100 --simec false --bg_window 1 --aot_override 0.1 --ozone true --aot_override 0.1 --ozone_override 0.33 --watervapor true --wv_override 2.0 --water_band B18 --water_threshold 0.06 --output_file /tmp/S3A_OL_1_EFR____20211205T042306_20211205T042606_20211205T060845_0179_079_204_2700_LN1_O_NR_002_processed.tif --sensor S3 --apply_gains false --glint_cor false --inlandwater false --productwater false --keep_land false --keep_water true --project false --working_folder /tmp/ /home/hello/E/Sentinel3/S3_CHL/S3A_OL_1_EFR____20211205T042306_20211205T042606_20211205T060845_0179_079_204_2700_LN1_O_NR_002/S3A_OL_1_EFR____20211205T042306_20211205T042606_20211205T060845_0179_079_204_2700_LN1_O_NR_002.SEN3/xfdumanifest.xml
Traceback (most recent call last):
File “/opt/vito/icor/src/icor.py”, line 15, in
import icor.sentinel2
File “/opt/vito/icor/src/icor/sentinel2.py”, line 12, in
from . import glintcor
File “/opt/vito/icor/src/icor/glintcor.py”, line 1, in
import numpy
ModuleNotFoundError: No module named ‘numpy’
Process exited with value 1
Finished tool execution in 1 seconds

[gnwiii]

@suthagar

import numpy
ModuleNotFoundError: No module named ‘numpy’

You probably need to install the numpy module. You haven’t provided much detail of your environment (linux distribution and version, python package (e.g., the python3 provided by your liinux distribution or anaconda, etc.). If you are using python3 provided by your linux distribution, you should try installing the numpy package using your distribution’s package manager. If you have already installed numpy then some deeper troubleshooting will be required.

[suthagar]

In my system i have Pycharm and now it show like this
opt/vito/icor/src/icor.py", line 15, in
import icor.sentinel2
File “/opt/vito/icor/src/icor/sentinel2.py”, line 12, in
from . import glintcor
File “/opt/vito/icor/src/icor/glintcor.py”, line 4, in
import rasterio
ModuleNotFoundError: No module named ‘rasterio’
Process exited with value 1
Finished tool execution in 0 seconds

If i need to install rasterio , where I need to install

[marpet]

The manual says that numpy, rasterio and gdal libraries are needed:

[suthagar]

thanks for your time…where I need to install above lib, I have Anoconda-spyder inside this I installed pandas, numpy, rasterio, even still error coming ,

or

I anything i need to inside SNAP?

[gnwiii]

Using Anaconda Python3 puts you in the “deeper” troubleshooting category. I have encountered this many times, and there are many variations on the problem and appropriate solution depending on your individual “use case”. The first step is to
work out the details of your current configuration.

The most common reason for:

@suthagar

ModuleNotFoundError: No module named ‘numpy’

is that ICOR is using the system /usr/bin/python3. Anaconda normally installs Python version 3 as just python, so you have two issues:

• invoking Anaconda Python 3 as python3
• ensuring that SNAP finds the Anaconda python3

Some users put conda activate ... in their shell startup scripts.

You can test your configuration by running snap from a terminal where python3 is Anaconda Python3.

First step: compare the output from python3 --version in the Ubuntu terminal with python3 --version or python --version in an Anaconda Terminal or after activating anaconda with conda activate in the Ubuntu terminal. A terminal where Anaconda is activated should display (base) at the start of the prompt. One you have a terminal with Anaconda Python3, you can check that the rasterio, numpy, and gdal packages are available with:

python[3] -c 'import <package>'

See Manage Anaconda Python for more details.

Normally, Anaconda Python 3 is run as just python. There are several ways to run Anaconda Python 3 using python3. The simplest is to create a symbolic link (ln -s python python3) in the directory that contains Anaconda’s python. Note that this can break some system scripts that expect python3 to be the system Python 3, so you need to be careful to limit the things you do when Anaconda Python 3 is active.

One you have a terminal where python3 gives Anaconda Python 3 and can load
the 3 required packages, you can try starting SNAP using:

<snap_install_path>/bin/snap

If this works you can continue to start SNAP from a terminal with Anaconda activated, or create a new Desktop Launcher by combining the one from SNAP with the example from: Desktop Launchers using Anaconda Python, but make sure you understand what the commands are doing.

[suthagar]

I tried this command

(base) hello@hello-Lenovo-Z50-70:~$ gdal-config --version

3.2.2

even icor is not run

File “/opt/vito/icor/src/icor.py”, line 15, in
import icor.sentinel2
File “/opt/vito/icor/src/icor/sentinel2.py”, line 14, in
import gdal
ModuleNotFoundError: No module named ‘gdal’
Process exited with value 1
Finished tool execution in 3 seconds

[gnwiii]

gdal-config is from the basic GDAL programs package, which is generally separate from the the python “module” in linux distros. Since you appear to be using Anaconda, you should install the gdal library using a conda package (mamba is usually faster than canda). Anaconda’s GDAL package may install different versions of the GDAL programs along with the Python module.

[suthagar]

Can I use this method, I tried but not run …

[suthagar]

I enter gdal-config in terminal
it show some info hereafter how to do

[gnwiii]

I don’t think the GDAL package from SNAP includes the Python module for your Anaconda Python. Like jpy, the Python gdal module has to be compiled for each platform and Python version. Unlike jpy, for most Python packages the gdal module is available from the same source (e.g., linux distro package or Anaconda).

[suthagar]

I changed that , also i try to install but it gdal already available ?please see this terminal

[gnwiii]

It would be easier to reply if you would just paste the text from the terminal into your post. Using
sudo pip install gdal will install the gdal Python module for your linux distribution’s package, but your previous post showed you using Anaconda Python. To install modules for Anacanda Python you need to use one of Anaconda’s package installers (conda or mamba) in an Anaconda environment (your post showed you using the “base” environment.

I don’t know anyone who doesn’t run into problems Python environments – see: xkcd on Python Environments

[suthagar]

But gdal in conda environment ? If avail why not run,
why my SNAP is not detect gdal?

Or its possible to clear this issue by uninstalling SNAP, Anaconda and reinstall one by one ?

[gnwiii]

Anaconda needs python -c 'from osgeo import gdal', so there are two issues using Anaconda Python3 with ICOR:

• python3 is called python
• the python GDAL module is found in OSGEO.

Is there a reason you need Anaconda Python, or can you use linux distro packages?

[suthagar]

Thanks for your time ,When i enter python or python3 in terminal it show version 3 and

, I tried this step (pip install gdal
) iit collect pakcages it say …

Building wheels for collected packages: gdal
Building wheel for gdal (setup.py) … error

Also I tried conda install gdal
it show

[gnwiii]

@suthagar

When i enter python or python3 in terminal it show version 3

There is more to getting a working configuration – your python and python3 may have different “minor” versions and could been built using different compiler and runtime library versions. You should avoid using distro versions of python and pip in an Anaconda environment.

Your first screen capture has “deprecated” messages errors that may indicate some mixture of versions of the various components (compilers, libraries, and python).

You appear to be using two different python programs (e.g., python3 from your linux distro and python from some other source. Using screen snapshots does not provide a complete picture, and they are almost unreadable on my laptop screen.

Please run type -a python python3 pip pip3 and then for each of these programs that are found, <full_path_of_program> --version and paste the text into a post as a “code” fragment using the </> format. This will help us understand which versions you are using.

You seem to be using pip. This should not be necessary, as both the linux distro and Anaconda should provide Python GDAL modules. Installing with other tools and may confuse things to point that it will not be possible for us to figure out a way to run ICOR from a distance.

If you must continue with pip, note that pip comes in different versions so you need to ensure that pip matches your python. Using <full_path_of_python_program> -m pip ... will ensure that the pip module matches your python program.

[suthagar]

Thank you dear gnwiii ,I tried your command

(base) hello@hello-Lenovo-Z50-70:~$ type -a python python3 pip pip3

python is /home/hello/anaconda3/bin/python
python3 is /home/hello/anaconda3/bin/python3
python3 is /usr/bin/python3
python3 is /bin/python3
pip is /home/hello/anaconda3/bin/pip
pip is /usr/bin/pip
pip is /bin/pip
pip3 is /home/hello/anaconda3/bin/pip3
pip3 is /usr/bin/pip3
pip3 is /bin/pip3

I further tried command as you suggested Output is below

(base) hello@hello-Lenovo-Z50-70:~ /home/hello/anaconda3/bin/python --version Python 3.9.7 (base) hello@hello-Lenovo-Z50-70:~ /home/hello/anaconda3/bin/python3 --version
Python 3.9.7
(base) hello@hello-Lenovo-Z50-70:~ /usr/bin/python3 --version Python 3.9.7 (base) hello@hello-Lenovo-Z50-70:~ /bin/python3 --version
Python 3.9.7

[suthagar]

also tried Using **<full_path_of_python_program> -m pip**

base) hello@hello-Lenovo-Z50-70:~$ /home/hello/anaconda3/bin/python -m pip

Usage:
/home/hello/anaconda3/bin/python -m pip [options]

Commands:
install Install packages.
download Download packages.
uninstall Uninstall packages.
freeze Output installed packages in requirements format.
list List installed packages.
show Show information about installed packages.
check Verify installed packages have compatible dependencies.
config Manage local and global configuration.
search Search PyPI for packages.
cache Inspect and manage pip’s wheel cache.
index Inspect information available from package indexes.
wheel Build wheels from your requirements.
hash Compute hashes of package archives.
completion A helper command used for command completion.
debug Show information useful for debugging.
help Show help for commands.

General Options:
-h, --help Show help.
–isolated Run pip in an isolated mode, ignoring
environment variables and user configuration.
-v, --verbose Give more output. Option is additive, and can be
used up to 3 times.
-V, --version Show version and exit.
-q, --quiet Give less output. Option is additive, and can be
used up to 3 times (corresponding to WARNING,
ERROR, and CRITICAL logging levels).
–log Path to a verbose appending log.
–no-input Disable prompting for input.
–proxy Specify a proxy in the form
[user:passwd@]proxy.server:port.
–retries Maximum number of retries each connection should
attempt (default 5 times).
–timeout Set the socket timeout (default 15 seconds).
–exists-action Default action when a path already exists:
(s)witch, (i)gnore, (w)ipe, (b)ackup, (a)bort.
–trusted-host Mark this host or host:port pair as trusted,
even though it does not have valid or any HTTPS.
–cert Path to PEM-encoded CA certificate bundle. If
provided, overrides the default. See ‘SSL
Certificate Verification’ in pip documentation
for more information.
–client-cert Path to SSL client certificate, a single file
containing the private key and the certificate
in PEM format.
–cache-dir Store the cache data in .
–no-cache-dir Disable the cache.
–disable-pip-version-check
Don’t periodically check PyPI to determine
whether a new version of pip is available for
download. Implied with --no-index.
–no-color Suppress colored output.
–no-python-version-warning
Silence deprecation warnings for upcoming
unsupported Pythons.
–use-feature Enable new functionality, that may be backward
incompatible.
–use-deprecated Enable deprecated functionality, that will be
removed in the future.

above result is again come when tried other s
/home/hello/anaconda3/bin/python
/home/hello/anaconda3/bin/python3
/usr/bin/python3
/bin/python3

                                        **and for pip** 

(base) hello@hello-Lenovo-Z50-70:~$ /bin/pip3 -m pip

Usage:
pip3 [options]

no such option: -m

I also checked with this command

(base) hello@hello-Lenovo-Z50-70:~ /home/hello/anaconda3/bin/python -m pip --version pip 21.2.4 from /home/hello/anaconda3/lib/python3.9/site-packages/pip (python 3.9) (base) hello@hello-Lenovo-Z50-70:~

[gnwiii]

Linux ICOR is for Ubuntu-20.04, and python3 -c ‘import gdal’ works for me in Ubuntu-20.04.

I did some snooping in the ICOR Python scripts from icor_install_ubuntu_20_04_x64_3.0.0.bin:

$ grep 'import.*gdal'  src/icor/*.py
src/icor/glintcor.py:    import gdal
src/icor/glintcor.py:    from osgeo import gdal
src/icor/sentinel2.py:import gdal

(note the indentation!) sentinel2.py uses import gdal but src/icor/glintcor.py uses a try-except block:

try:
    import gdal
except ImportError:
    from osgeo import gdal

You could try editing sentinel2.py to replace import gdal with the try-except block.

[I notice a bunch of <Control-C> `^C` entries – if you were trying copy text you need to use <ShiftControl-C> to copy text from the terminal.]