Trying to run gpt command creates a syntax error that does not appear to exist

development python
#1

Describe the bug

When running the code below in Pycharm, the code successfully works when simply selecting these lines of code and then running them (Shift+Alt+E). But when running the code by pressing Shift+F10 or the “Run” button, there is a syntax error that does not appear to exist on the line of code that includes the gpt command. Have you come across this problem before? If so, is it related to the gpt package or to the Python compiler? If it is the compiler, I can try asking about this error on a Python-specific forum also. This code is related to the code provided in the link below but the S1 image querying and downloading has already been completed.

Thank you in advance

Kind regards

Craig Pearce

Subset.xml (1.5 KB)

https://eodag.readthedocs.io/en/stable/notebooks/tutos/tuto_ship_detection.html

Code To Reproduce

# Import packages
import os

# Set base working directory
base_wd = '/home/craig/PycharmProjects/S1_image_processing'

# Add absolute path to SNAP bin folder to make the gpt command available
os.environ["PATH"] = "/home/craig/snap/bin" + ":" + os.environ["PATH"]

# Location of xml file
graph_subset = '/home/craig/PycharmProjects/S1_image_processing/eodag_workspace_shipdetection/xml_files/Subset.xml'

# Set path to unzipped Sentinel-1 image folder
product_path = '/home/craig/PycharmProjects/S1_image_processing/eodag_workspace_shipdetection/S1_images/S1B_IW_GRDH_1SDV_20210502T051838_20210502T051903_026721_03311D_6AAA.SAFE'

# Set extent of image in pixels (i.e. do not subset image by pixels)
coords_str = '0,0,16704,26119'

# Set location of shapefile
shapefile_loc = '/home/craig/PycharmProjects/Shapefiles/Natural_Earth/ne_10m_ocean_fix_singlepart.shp'

# Set output location
output1 = '/home/craig/PycharmProjects/S1_image_processing/eodag_workspace_shipdetection/outputs/S1B_IW_GRDH_1SDV_20210502T051838_6AAA/S1B_IW_GRDH_1SDV_20210502T051838_6AAA_Orb'

# Check that files exist
print(os.path.isfile(graph_subset))
print(os.path.isdir(product_path))
print(os.path.isfile(shapefile_loc))

!gpt {graph_subset} -Pinputproduct={product_path} -Pinputcoords={coords_str} -Pvectorfile={shapefile_loc} -Poutputproduct={output1}

Output

When running selected lines in Pycharm (Shift+Alt+E)

True
True
True
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Incompatible GDAL 3.3.2 found on system. Internal GDAL 3.0.0 from distribution will be used.
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Internal GDAL 3.0.0 set to be used by SNAP.
INFO: org.esa.snap.core.util.EngineVersionCheckActivator: Please check regularly for new updates for the best SNAP experience.
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Internal GDAL 3.0.0 set to be used by SNAP.
Executing processing graph
INFO: org.hsqldb.persist.Logger: dataFileCache open start
…10%…20%…30%…40%…50%…60%…70%…80%…90% done.

When running code in Pycharm (Shift+F10)

Error points to exclamation mark

!gpt {graph_subset} -Pinputproduct={product_path} -Pinputcoords={coords_str} -Pvectorfile={shapefile_loc} -Poutputproduct={output1}

SyntaxError: invalid syntax

Environment:

  • Python version: 3.8.10 (default, Nov 26 2021, 20:14:08) \n[GCC 9.3.0]
  • PyCharm 2021.3.2 (Community Edition)
    Build #PC-213.6777.52, built on January 28, 2022
    Runtime version: 11.0.13+7-b1751.25 amd64
    VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
    Linux 5.13.0-27-generic
    GC: G1 Young Generation, G1 Old Generation
    Memory: 2048M
    Cores: 12
    Non-Bundled Plugins:
    R4Intellij (213.5744.6)
    Current Desktop: ubuntu:GNOME
#2

The exclamation mark is exclusive for Jupyter notebooks.
What’s the Meaning of the Exclamation Mark in a Jupyter Notebook? – Finxter
It lets you execute commands from the underlying operating system.
Probably the two execution modes you are mentioning make the difference.

2 Likes
#3

Thank you @marpet for your prompt response. I wondered if this was the issue but did not think that the exclamation mark is exclusive to Jupyter notebook. Would it be helpful for me to find a solution then post it here?

Update - The solution is as follows:

instead of using: !gpt {graph_process} -Pinputproduct={os.path.join(base_wd, workspace, output_dir, img_savename, gpt_input)} -Pvectorfileshort={shapefile_name} -Poutputproduct1={os.path.join(base_wd, workspace, output_dir, img_savename, gpt_output)} -Poutputproduct2={os.path.join(base_wd, workspace, output_dir, img_savename, gpt_output)}

Use: command = “gpt " + graph_subset + " -Pinputproduct=” + product_path + " -Pinputcoords=" + coords_str + " -Pvectorfile=" + shapefile_loc + " -Poutputproduct=" + os.path.join(base_wd, workspace, output_dir, img_savename, gpt_subset_filename)
os.system(command)

1 Like