Issues with GPT via Python’s subprocess.run()

development python
#1

(Apologies with the crosspost here, but this question is probably best posted in this forum)

Hi,

I can successfully run GPT via command line:

(eodag) C:\Users\User\Documents\SARShipDetection>gpt workspace\SNAP_Preprocessing.xml -Pinputproduct=workspace\S1A_IW_GRDH_1SDV_20170501T165806_20170501T165831_016391_01B235_9CD1.zip -Pvectorfile=workspace\auxdata\Gulf_of_Trieste_seamask_UTM33.shp -Poutputproduct=workspace\outputs\S1A_IW_GRDH_1SDV_20161009T165807_4550_Orb
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.esa.snap.core.util.EngineVersionCheckActivator: Please check regularly for new updates for the best SNAP experience.
Executing processing graph
INFO: org.hsqldb.persist.Logger: dataFileCache open start
....10%....20%....30%....40%....50%....60%....70%....80%....90% done.

However, if I try to run the above using subprocess.run() in a Python script:

graph_preprocess = "workspace\\SNAP_Preprocessing.xml"
product_selected_path = "workspace\\S1A_IW_GRDH_1SDV_20170501T165806_20170501T165831_016391_01B235_9CD1.zip"
vector_filepath = "workspace\\auxdata\\Gulf_of_Trieste_seamask_UTM33.shp"
output_dir = "workspace\\outputs"
outfn_preprocessed = "S1A_IW_GRDH_1SDV_20161009T165807_4550_Orb"

subprocess.run(["gpt", graph_preprocess, "-Pinputproduct=", product_selected_path, "-Pvectorfile=", vector_filepath, "-Poutputproduct=", os.path.join(output_dir, outfn_preprocessed)], stdout=subprocess.PIPE)

I get the following output with a returncode=1:

CompletedProcess(args=['gpt', 'workspace\\SNAP_Preprocessing.xml', '-Pinputproduct=', 'workspace\\S1A_IW_GRDH_1SDV_20170501T165806_20170501T165831_016391_01B235_9CD1.zip', '-Pvectorfile=', 'workspace\\auxdata\\Gulf_of_Trieste_seamask_UTM33.shp', '-Poutputproduct=', 'workspace\\outputs\\S1A_IW_GRDH_1SDV_20161009T165807_4550_Orb'], returncode=1, stdout=b'Executing processing graph\r\n done.\r\n')
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.esa.snap.core.util.EngineVersionCheckActivator: Please check regularly for new updates for the best SNAP experience.

Error: [NodeId: Read] The 'file' parameter is not set

If I understand the error, GPT run via Python’s subprocess.run() cannot find the ‘file’ parameter in supplied .xml file (workspace\\SNAP_Preprocessing.xml)? If this interpretation is correct, why can it find this parameter when running from the command line? Is there some nuance about running GPT via Python’s subprocess.run() that I’m missing?

Any advice would be greatly appreciated!

Cheers,

Dennis

SNAP_Preprocessing.xml (1.2 KB)

PS - If I substitute the subprocess.run() line in the above Python script with a properly formatted IPython Bang (!) command, the processing completes successfully:

!gpt {graph_preprocess} -Pinputproduct={product_selected_path} -Pvectorfile={vector_filepath} -Poutputproduct={os.path.join(output_dir, outfn_preprocessed)}

Development Environment:
Python 3.11.3
eodag 2.10.0 pyhd8ed1ab_0 conda-forge
eodag-cube 0.3.0 pypi_0 pypi
Windows 10 (unfortunately…)
SNAP Desktop 9.0.6
SNAP Engine 9.0.6
OpenJDK Runtime Environment 1.8.0_242-b20
OpenJDK 64-bit Server VM by Azul Systems, Inc.
SNAP Desktop 9.0.6
SNAP Engine 9.0.6
OpenJDK Runtime Environment 1.8.0_242-b20
OpenJDK 64-bit Server VM by Azul Systems, Inc.

#2

For the moment, I’ve found an ugly hack: programmatically create a Windows .bat file that contains the command and all necessary variables and paths. In my case, the batch file (process.bat') contained a single (long) line representing the command line input:

gpt workspace\SNAP_Preprocessing.xml -Pinputproduct=workspace\inputs\S1A_IW_GRDH_1SDV_20170501T165806_20170501T165831_016391_01B235_9CD1.zip -Pvectorfile=workspace\auxdata\Gulf_of_Trieste_seamask_UTM33.shp -Poutputproduct=workspace\outputs\S1A_IW_GRDH_1SDV_20161009T165807_4550_Orb

One can then call subprocess.run("process.bat") in the Python script :roll_eyes:

It works, but it’s not an ideal solution. Hopefully someone with more experience can suss out the actual issue.

Cheers

1 Like
#3

Hi @dduro try this:

subprocess.run(["gpt", graph_preprocess, f"-Pinputproduct={product_selected_path}", f"-Pvectorfile={vector_filepath}", f"-Poutputproduct={os.path.join(output_dir, outfn_preprocessed)}"], stdout=subprocess.PIPE)

Or just give pyroSAR a try :wink:
https://pyrosar.readthedocs.io/en/latest/general/snap.html

1 Like
#4

Danke! That definitely did the trick @johntruckenbrodt !

Looks like I should brush up on Python’s “F-strings”, but pyroSAR looks like an interesting alternative as well. I’ll definitely check it out.

Cheers

#5

Another working solution mentioned by @cndnflyr here is to properly concatenate the strings using the “+” operator. This solution also works, although use of a formatted string literal is easier to read IMHO, and there appears to be other benefits associated with this approach.

1 Like