Python snappy sen2cor plugin

[christos19004]

hello

i want to use sen2cor plugin in standalone python script .
I use linux and python 3 and i have snap 8 with sen2cor plugin

first I tried using command line :

/home/username/.snap/auxdata/Sen2Cor-02.08.00-Linux64/bin/L2A_Process --help
usage: L2A_Process.py [-h] [--mode MODE] [--resolution {10,20,60}]
                      [--datastrip DATASTRIP] [--tile TILE]
                      [--output_dir OUTPUT_DIR] [--work_dir WORK_DIR]
                      [--img_database_dir IMG_DATABASE_DIR]
                      [--res_database_dir RES_DATABASE_DIR]
                      [--processing_centre PROCESSING_CENTRE]
                      [--archiving_centre ARCHIVING_CENTRE]
                      [--processing_baseline PROCESSING_BASELINE] [--raw]
                      [--tif] [--sc_only] [--cr_only] [--debug]
                      [--GIP_L2A GIP_L2A] [--GIP_L2A_SC GIP_L2A_SC]
                      [--GIP_L2A_AC GIP_L2A_AC] [--GIP_L2A_PB GIP_L2A_PB]
                      input_dir

Sentinel-2 Level 2A Processor (Sen2Cor). Version: 2.8.0, created: 2019.02.20,
supporting Level-1C product version 14.2 - 14.5.

positional arguments:
  input_dir             Directory of Level-1C input

optional arguments:
  -h, --help            show this help message and exit
  --mode MODE           Mode: generate_datastrip, process_tile
  --resolution {10,20,60}
                        Target resolution, can be 10, 20 or 60m. If omitted,
                        only 20 and 10m resolutions will be processed
  --datastrip DATASTRIP
                        Datastrip folder
  --tile TILE           Tile folder
  --output_dir OUTPUT_DIR
                        Output directory
  --work_dir WORK_DIR   Work directory
  --img_database_dir IMG_DATABASE_DIR
                        Database directory for L1C input images
  --res_database_dir RES_DATABASE_DIR
                        Database directory for results and temporary products
  --processing_centre PROCESSING_CENTRE
                        Processing centre as regex: ^[A-Z_]{4}$, e.g "SGS_"
  --archiving_centre ARCHIVING_CENTRE
                        Archiving centre as regex: ^[A-Z_]{4}$, e.g. "SGS_"
  --processing_baseline PROCESSING_BASELINE
                        Processing baseline in the format: "dd.dd", where
                        d=[0:9]
  --raw                 Export raw images in rawl format with ENVI hdr
  --tif                 Export raw images in TIFF format instead of JPEG-2000
  --sc_only             Performs only the scene classification at 60 or 20m
                        resolution
  --cr_only             Performs only the creation of the L2A product tree, no
                        processing
  --debug               Performs in debug mode
  --GIP_L2A GIP_L2A     Select the user GIPP
  --GIP_L2A_SC GIP_L2A_SC
                        Select the scene classification GIPP
  --GIP_L2A_AC GIP_L2A_AC
                        Select the atmospheric correction GIPP
  --GIP_L2A_PB GIP_L2A_PB
                        Select the processing baseline GIP

and this python code without success:

from snappy import GPF, HashMap, Rectangle
from snappy import ProductIO
cmd='/home/username/.snap/auxdata/Sen2Cor-02.08.00-Linux64/bin/L2A_Process /home/username/path/to.SAFE --resolution=10'
os.system(cmd)

error message :

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.thoughtworks.xstream.core.util.Fields (file:/home/username/snap/snap/modules/ext/org.esa.snap.ceres-core/com-thoughtworks-xstream/xstream.jar) to field java.util.TreeMap.comparator
WARNING: Please consider reporting this to the maintainers of com.thoughtworks.xstream.core.util.Fields
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
INFO: org.esa.snap.python.gpf.PyOperatorSpi: Python operator 'py_sambuca_snap_op' registered (Python module: 'sambuca_snap_op', class: 'sambuca_snap_op', root: '/home/username/.snap/system/modules/org-esa-sen2coral-sen2coral-inversion.jar')
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Incompatible GDAL 3.2.1 found on system. Internal GDAL 3.0.0 from distribution will be used.
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Internal GDAL 3.0.0 set to be used by SNAP.
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.esa.snap.core.util.EngineVersionCheckActivator: Please check regularly for new updates for the best SNAP experience.
Syntax error in metadata, see report file for details.
Schema file: S2-PDGS-TAS-DI-PSD-V14.5_Schema/S2_PDI_Level-1C_Datastrip_Metadata.xsd
Details: Element '{https://psd-14.sentinel2.eo.esa.int/PSD/S2_PDI_Level-2A_Datastrip_Metadata.xsd}Level-2A_DataStrip_ID': No matching global declaration available for the validation root.
to.SAFE: identifier S2?_MSIL1C*.SAFE is missing.
Traceback (most recent call last):
  File "/home/username/.snap/auxdata/Sen2Cor-02.08.00-Linux64/lib/python2.7/site-packages/sen2cor/L2A_Process.py", line 534, in <module>
    sys.exit(main())
  File "/home/username/.snap/auxdata/Sen2Cor-02.08.00-Linux64/lib/python2.7/site-packages/sen2cor/L2A_Process.py", line 383, in main
    L2A_UP_ID =  os.path.join(config.logDir, config.L2A_UP_ID)
  File "/home/username/.snap/auxdata/Sen2Cor-02.08.00-Linux64/lib/python2.7/posixpath.py", line 68, in join
    if b.startswith('/'):
AttributeError: 'NoneType' object has no attribute 'startswith'

second attempt i tried to using python code without success:

import snappy
from snappy import ProductIO
from snappy import GPF
from snappy import jpy
from snappy import HashMap
s=ProductIO.readProduct('/home/username/Desktop/path/to.SAFE')
HashMap = snappy.jpy.get_type('java.util.HashMap')
snappy.GPF.getDefaultInstance().getOperatorSpiRegistry().loadOperatorSpis()
parameters = HashMap()
parameters.put('resolution', '10')
L2A_image = snappy.GPF.createProduct('Sen2Cor280', parameters, s)




WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.thoughtworks.xstream.core.util.Fields (file:/home/username/snap/snap/modules/ext/org.esa.snap.ceres-core/com-thoughtworks-xstream/xstream.jar) to field java.util.TreeMap.comparator
WARNING: Please consider reporting this to the maintainers of com.thoughtworks.xstream.core.util.Fields
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
INFO: org.esa.snap.python.gpf.PyOperatorSpi: Python operator 'py_sambuca_snap_op' registered (Python module: 'sambuca_snap_op', class: 'sambuca_snap_op', root: '/home/username/.snap/system/modules/org-esa-sen2coral-sen2coral-inversion.jar')
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Incompatible GDAL 3.2.1 found on system. Internal GDAL 3.0.0 from distribution will be used.
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Internal GDAL 3.0.0 set to be used by SNAP.
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.esa.snap.core.util.EngineVersionCheckActivator: Please check regularly for new updates for the best SNAP experience.
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Internal GDAL 3.0.0 set to be used by SNAP.
INFO: org.esa.s2tbx.dataio.s2.ortho.S2OrthoProductReaderPlugIn: Building product reader - EPSG:32634
WARNING: org.esa.snap.core.metadata.GenericXmlMetadata: Metadata: the path to element [metadata_level] does not exist
WARNING: org.esa.snap.core.metadata.GenericXmlMetadata: Metadata: the path to element [bandid] does not exist
WARNING: org.esa.snap.core.metadata.GenericXmlMetadata: Metadata: the path to element [bandid] does not exist
WARNING: org.esa.snap.core.metadata.GenericXmlMetadata: Metadata: the path to element [bandid] does not exist
WARNING: org.esa.snap.core.metadata.GenericXmlMetadata: Metadata: the path to element [bandid] does not exist
WARNING: org.esa.snap.core.metadata.GenericXmlMetadata: Metadata: the path to element [bandid] does not exist
INFO: org.hsqldb.persist.Logger: dataFileCache open start
WARNING: org.esa.s2tbx.dataio.s2.ortho.metadata.S2OrthoMetadata: Warning: missing file home/username/Desktop/path/to.SAFE/GRANULE/L2A_T34TFL_A033695_20211204T092346/QI_DATA/L2A_T34TFL_20211204T092351_DDV_20m.jp2

WARNING: org.esa.s2tbx.dataio.s2.ortho.metadata.S2OrthoMetadata: Warning: no image files found for band quality_dense_dark_vegetation

INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterOp: Executing tool 'Sen2Cor280' with command line: '/home/username/.snap/auxdata/Sen2Cor-02.08.00-Linux64/bin/L2A_Process --resolution 10 home/username/Desktop/path/to.SAFE --GIP_L2A /home/username/.snap/auxdata/Sen2Cor-02.08.00-Linux64/L2A-GIPP28-custom.xml'
INFO: org.esa.snap.core.gpf.operators.tooladapter.DefaultOutputConsumer: Syntax error in metadata, see report file for details.
INFO: org.esa.snap.core.gpf.operators.tooladapter.DefaultOutputConsumer: Schema file: S2-PDGS-TAS-DI-PSD-V14.5_Schema/S2_PDI_Level-1C_Datastrip_Metadata.xsd
INFO: org.esa.snap.core.gpf.operators.tooladapter.DefaultOutputConsumer: Details: Element '{https://psd-14.sentinel2.eo.esa.int/PSD/S2_PDI_Level-2A_Datastrip_Metadata.xsd}Level-2A_DataStrip_ID': No matching global declaration available for the validation root.
INFO: org.esa.snap.core.gpf.operators.tooladapter.DefaultOutputConsumer: to.SAFE: identifier S2?_MSIL1C*.SAFE is missing.
INFO: org.esa.snap.core.gpf.operators.tooladapter.DefaultOutputConsumer: Traceback (most recent call last):
INFO: org.esa.snap.core.gpf.operators.tooladapter.DefaultOutputConsumer:   File "/home/username/.snap/auxdata/Sen2Cor-02.08.00-Linux64/lib/python2.7/site-packages/sen2cor/L2A_Process.py", line 534, in <module>
INFO: org.esa.snap.core.gpf.operators.tooladapter.DefaultOutputConsumer:     sys.exit(main())
INFO: org.esa.snap.core.gpf.operators.tooladapter.DefaultOutputConsumer:   File "/home/username/.snap/auxdata/Sen2Cor-02.08.00-Linux64/lib/python2.7/site-packages/sen2cor/L2A_Process.py", line 383, in main
INFO: org.esa.snap.core.gpf.operators.tooladapter.DefaultOutputConsumer:     L2A_UP_ID =  os.path.join(config.logDir, config.L2A_UP_ID)
INFO: org.esa.snap.core.gpf.operators.tooladapter.DefaultOutputConsumer:   File "/home/username/.snap/auxdata/Sen2Cor-02.08.00-Linux64/lib/python2.7/posixpath.py", line 68, in join
INFO: org.esa.snap.core.gpf.operators.tooladapter.DefaultOutputConsumer:     if b.startswith('/'):
SEVERE: org.esa.snap.core.gpf.operators.tooladapter.DefaultOutputConsumer: AttributeError: 'NoneType' object has no attribute 'startswith'
INFO: org.esa.snap.core.gpf.operators.tooladapter.DefaultOutputConsumer: Process exited with value 1
INFO: org.esa.snap.core.gpf.operators.tooladapter.DefaultOutputConsumer: Finished tool execution in 3 seconds
INFO: org.esa.s2tbx.dataio.s2.ortho.S2OrthoProductReaderPlugIn: Building product reader - EPSG:32634
WARNING: org.esa.snap.core.metadata.GenericXmlMetadata: Metadata: the path to element [metadata_level] does not exist
WARNING: org.esa.snap.core.metadata.GenericXmlMetadata: Metadata: the path to element [bandid] does not exist
WARNING: org.esa.snap.core.metadata.GenericXmlMetadata: Metadata: the path to element [bandid] does not exist
WARNING: org.esa.snap.core.metadata.GenericXmlMetadata: Metadata: the path to element [bandid] does not exist
WARNING: org.esa.snap.core.metadata.GenericXmlMetadata: Metadata: the path to element [bandid] does not exist
WARNING: org.esa.snap.core.metadata.GenericXmlMetadata: Metadata: the path to element [bandid] does not exist
WARNING: org.esa.s2tbx.dataio.s2.ortho.metadata.S2OrthoMetadata: Warning: missing file home/username/Desktop/path/to.SAFE/GRANULE/L2A_T34TFL_A033695_20211204T092346/QI_DATA/L2A_T34TFL_20211204T092351_DDV_20m.jp2

WARNING: org.esa.s2tbx.dataio.s2.ortho.metadata.S2OrthoMetadata: Warning: no image files found for band quality_dense_dark_vegetation

100% done.

here sen2cor system variables :

SEN2COR28_BIN				$HOME/.snap/auxdata/Sen2Cor-02.08.00-Linux64
SEN2COR_EXE				/bin/L2A_Process
USERPROFILE					$HOME
HOME								/home/username
USER_HOME					$HOME

anyd idea how can i using sen2cor plugin in python code ?

[marpet]

Using snappy and the tool adapter should work but it is just an indirection.

I think the best way to use sen2cor from Python is explained in the
[Sen2Cor Configuration and User Manual - V2.10.0 (esa.int)] (http://step.esa.int/thirdparties/sen2cor/2.10.0/docs/S2-PDGS-MPC-L2A-SUM-V2.10.0.pdf) section 3.4 The Software Development Environment.

To use it directly as API was als requested several times. But I think this is still not possible.

Accessing sen2cor scene classification from Python

Hi, I would like to include sen2cor scene classification in my own processing chain in Python. I could run the command line interface L2A_Process with some arguments through subprocess and try to interpret the output. But it would be much better to have it all happen within my Python session. I guess I could build config like it is done in L2A_Process and then feed that to L2A_SceneClass, doing without all of L2A_ProcessTile and L2A_Schedule. I can see that it is easy to generate tables from…

Sen2cor and python module

I have installed the last release of Sen2Cor in my windows system , including it in the Anaconda environment so that now sen2cor is also set as a python package for python. Currently Sen2Cor can run only if launched in the command window, but is it possible to import sen2cor as a python library in aa script and to use it in this way: a = sen2cor(arguments) where among the arguments there is for example an object of class Product from the snappy module? It would be useful to use it in a proc…

It seems sen2cor is still using Python 2.7. This can cause conflicts if you use Python 3.

[christos19004]

How to configure python 3 in sen2cor ?where is the best option to use python sen2cor plugin?

Thank you