Error on executing graph

Good afternoon to all!
I am experiencing -I believe- a similar issue to some alreasy shared before. The source file printed with echo in a loop, shows that it is correctly identified. Any clues as to why this error (Error: [NodeId: Read] Specified ‘file’ [${sourceFile}] does not exist.) occurs would be greatly appreciated!

Processing Y:\a-e_2020_whk\07\D_code\Downloads_2022-03-25_to_2022-04-03_ASCENDING\IT-Bon\S1A_IW_GRDH_1SDV_20220331T164839_20220331T164904_042568_0513E0_0657.SAFE
targetFile …..\Downloads_2022-03-25_to_2022-04-03_ASCENDING\IT-Bon_processed\processed_S1A_IW_GRDH_1SDV_20220331T164839_20220331T164904_042568_0513E0_0657.dim
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.esa.snap.core.util.EngineVersionCheckActivator: Please check regularly for new updates for the best SNAP experience.
Executing processing graph
INFO: org.esa.s1tbx.commons.io.ImageIOFile: Using FileCacheImageInputStream
INFO: org.esa.s1tbx.commons.io.ImageIOFile: Using FileCacheImageInputStream
INFO: org.hsqldb.persist.Logger: dataFileCache open start
done.
org.esa.snap.core.gpf.graph.GraphException: [NodeId: Read] Specified ‘file’ [${sourceFile}] does not exist.
at org.esa.snap.core.gpf.graph.NodeContext.initTargetProduct(NodeContext.java:79)
at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:199)
at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:182)
at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:182)
at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:182)
at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:182)
at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:182)
at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:182)
at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:182)
at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:182)
at org.esa.snap.core.gpf.graph.GraphContext.initOutput(GraphContext.java:166)
at org.esa.snap.core.gpf.graph.GraphContext.(GraphContext.java:85)
at org.esa.snap.core.gpf.graph.GraphContext.(GraphContext.java:58)
at org.esa.snap.core.gpf.graph.GraphProcessor.executeGraph(GraphProcessor.java:127)
at org.esa.snap.core.gpf.main.DefaultCommandLineContext.executeGraph(DefaultCommandLineContext.java:86)
at org.esa.snap.core.gpf.main.CommandLineTool.executeGraph(CommandLineTool.java:547)
at org.esa.snap.core.gpf.main.CommandLineTool.runGraph(CommandLineTool.java:391)
at org.esa.snap.core.gpf.main.CommandLineTool.runGraphOrOperator(CommandLineTool.java:287)
at org.esa.snap.core.gpf.main.CommandLineTool.run(CommandLineTool.java:188)
at org.esa.snap.core.gpf.main.CommandLineTool.run(CommandLineTool.java:121)
at org.esa.snap.core.gpf.main.GPT.run(GPT.java:59)
at org.esa.snap.core.gpf.main.GPT.main(GPT.java:37)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.esa.snap.runtime.Launcher.lambda$run$0(Launcher.java:55)
at org.esa.snap.runtime.Engine.runClientCode(Engine.java:189)
at org.esa.snap.runtime.Launcher.run(Launcher.java:51)
at org.esa.snap.runtime.Launcher.main(Launcher.java:31)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.exe4j.runtime.LauncherEngine.launch(LauncherEngine.java:84)
at com.exe4j.runtime.WinLauncher.main(WinLauncher.java:94)
at com.install4j.runtime.launcher.WinLauncher.main(WinLauncher.java:25)
Caused by: org.esa.snap.core.gpf.OperatorException: Specified ‘file’ [${sourceFile}] does not exist.
at org.esa.snap.core.gpf.common.ReadOp.initialize(ReadOp.java:121)
at org.esa.snap.core.gpf.internal.OperatorContext.initializeOperator(OperatorContext.java:528)
at org.esa.snap.core.gpf.internal.OperatorContext.getTargetProduct(OperatorContext.java:298)
at org.esa.snap.core.gpf.Operator.getTargetProduct(Operator.java:385)
at org.esa.snap.core.gpf.graph.NodeContext.initTargetProduct(NodeContext.java:77)
… 36 more

Error: [NodeId: Read] Specified ‘file’ [${sourceFile}] does not exist.

Please provide also your graph xml file and the batch file you are using. Also, the command to invoke the script would be helpful.

A guess is that you are specifying the source file with -SsourceFile= instead of -PsourceFile=.

command to invoke the script:


.\processDataset.bat graph_flats_sigma.xml s1_preprocessing.properties ..\..\Downloads_2022-03-25_to_2022-04-03_ASCENDING\IT-Bon ..\..\Downloads_2022-03-25_to_2022-04-03_ASCENDING\IT-Bon_processed processed

graph xml file:

<graph id="Graph">
  <version>1.0</version>
  <node id="Read">
    <operator>Read</operator>
    <sources/>
    <parameters class="com.bc.ceres.binding.dom.XppDomElement">
      <file>${sourceFile}</file>
    </parameters>
  </node>
  <node id="Apply-Orbit-File">
    <operator>Apply-Orbit-File</operator>
    <sources>
      <sourceProduct refid="Read"/>
    </sources>
    <parameters class="com.bc.ceres.binding.dom.XppDomElement">
      <orbitType>Sentinel Precise (Auto Download)</orbitType>
      <polyDegree>3</polyDegree>
      <continueOnFail>true</continueOnFail>
    </parameters>
  </node>
  <node id="Calibration">
    <operator>Calibration</operator>
    <sources>
      <sourceProduct refid="Apply-Orbit-File"/>
    </sources>
    <parameters class="com.bc.ceres.binding.dom.XppDomElement">
      <sourceBands/>
      <auxFile>Product Auxiliary File</auxFile>
      <externalAuxFile/>
      <outputImageInComplex>false</outputImageInComplex>
      <outputImageScaleInDb>false</outputImageScaleInDb>
      <createGammaBand>false</createGammaBand>
      <createBetaBand>false</createBetaBand>
      <selectedPolarisations>VH,VV</selectedPolarisations>
      <outputSigmaBand>true</outputSigmaBand>
      <outputGammaBand>false</outputGammaBand>
      <outputBetaBand>false</outputBetaBand>
    </parameters>
  </node>
  <node id="Speckle-Filter">
    <operator>Speckle-Filter</operator>
    <sources>
      <sourceProduct refid="Calibration"/>
    </sources>
    <parameters class="com.bc.ceres.binding.dom.XppDomElement">
      <sourceBands>Sigma0_VH,Sigma0_VV</sourceBands>
      <filter>Lee Sigma</filter>
      <filterSizeX>3</filterSizeX>
      <filterSizeY>3</filterSizeY>
      <dampingFactor>2</dampingFactor>
      <estimateENL>true</estimateENL>
      <enl>1.0</enl>
      <numLooksStr>1</numLooksStr>
      <windowSize>7x7</windowSize>
      <targetWindowSizeStr>3x3</targetWindowSizeStr>
      <sigmaStr>0.9</sigmaStr>
      <anSize>50</anSize>
    </parameters>
  </node>
  <node id="Terrain-Correction">
    <operator>Terrain-Correction</operator>
    <sources>
      <sourceProduct refid="Speckle-Filter"/>
    </sources>
    <parameters class="com.bc.ceres.binding.dom.XppDomElement">
      <sourceBands>Sigma0_VH,Sigma0_VV</sourceBands>
      <demName>SRTM 1Sec HGT</demName>
      <externalDEMFile/>
      <externalDEMNoDataValue>0.0</externalDEMNoDataValue>
      <externalDEMApplyEGM>true</externalDEMApplyEGM>
      <demResamplingMethod>BILINEAR_INTERPOLATION</demResamplingMethod>
      <imgResamplingMethod>BILINEAR_INTERPOLATION</imgResamplingMethod>
      <pixelSpacingInMeter>${resolution}</pixelSpacingInMeter>
      <pixelSpacingInDegree>2.6949458523585647E-4</pixelSpacingInDegree>
      <mapProjection>GEOGCS[&quot;WGS84(DD)&quot;, &#xd;
  DATUM[&quot;WGS84&quot;, &#xd;
    SPHEROID[&quot;WGS84&quot;, 6378137.0, 298.257223563]], &#xd;
  PRIMEM[&quot;Greenwich&quot;, 0.0], &#xd;
  UNIT[&quot;degree&quot;, 0.017453292519943295], &#xd;
  AXIS[&quot;Geodetic longitude&quot;, EAST], &#xd;
  AXIS[&quot;Geodetic latitude&quot;, NORTH]]</mapProjection>
      <alignToStandardGrid>false</alignToStandardGrid>
      <standardGridOriginX>0.0</standardGridOriginX>
      <standardGridOriginY>0.0</standardGridOriginY>
      <nodataValueAtSea>true</nodataValueAtSea>
      <saveDEM>false</saveDEM>
      <saveLatLon>false</saveLatLon>
      <saveIncidenceAngleFromEllipsoid>false</saveIncidenceAngleFromEllipsoid>
      <saveLocalIncidenceAngle>true</saveLocalIncidenceAngle>
      <saveProjectedLocalIncidenceAngle>false</saveProjectedLocalIncidenceAngle>
      <saveSelectedSourceBand>true</saveSelectedSourceBand>
      <outputComplex>false</outputComplex>
      <applyRadiometricNormalization>false</applyRadiometricNormalization>
      <saveSigmaNought>false</saveSigmaNought>
      <saveGammaNought>false</saveGammaNought>
      <saveBetaNought>false</saveBetaNought>
      <incidenceAngleForSigma0>Use projected local incidence angle from DEM</incidenceAngleForSigma0>
      <incidenceAngleForGamma0>Use projected local incidence angle from DEM</incidenceAngleForGamma0>
      <auxFile>Latest Auxiliary File</auxFile>
      <externalAuxFile/>
    </parameters>
  </node>
  <node id="BandMerge">
    <operator>BandMerge</operator>
    <sources>
      <sourceProduct refid="LinearToFromdB"/>
      <sourceProduct.1 refid="BandSelect"/>
    </sources>
    <parameters class="com.bc.ceres.binding.dom.XppDomElement">
      <sourceBands/>
      <geographicError>1.0E-5</geographicError>
    </parameters>
  </node>
  <node id="BandSelect">
    <operator>BandSelect</operator>
    <sources>
      <sourceProduct refid="Terrain-Correction"/>
    </sources>
    <parameters class="com.bc.ceres.binding.dom.XppDomElement">
      <selectedPolarisations/>
      <sourceBands/>
      <bandNamePattern/>
    </parameters>
  </node>
  <node id="LinearToFromdB">
    <operator>LinearToFromdB</operator>
    <sources>
      <sourceProduct refid="Terrain-Correction"/>
    </sources>
    <parameters class="com.bc.ceres.binding.dom.XppDomElement">
      <sourceBands>Sigma0_VH,Sigma0_VV</sourceBands>
    </parameters>
  </node>
  <node id="Subset">
    <operator>Subset</operator>
    <sources>
      <sourceProduct refid="BandMerge"/>
    </sources>
    <parameters class="com.bc.ceres.binding.dom.XppDomElement">
      <sourceBands>Sigma0_VH_db,Sigma0_VV_db,localIncidenceAngle</sourceBands>
      <region>0,0,0,0</region>
      <referenceBand/>
      <geoRegion>${wkt}</geoRegion>
      <subSamplingX>1</subSamplingX>
      <subSamplingY>1</subSamplingY>
      <fullSwath>false</fullSwath>
      <copyMetadata>true</copyMetadata>
    </parameters>
  </node>
  <applicationData id="Presentation">
    <Description/>
    <node id="Read">
            <displayPosition x="37.0" y="134.0"/>
    </node>
    <node id="Apply-Orbit-File">
      <displayPosition x="75.0" y="171.0"/>
    </node>
    <node id="Calibration">
      <displayPosition x="148.0" y="209.0"/>
    </node>
    <node id="Terrain-Flattening">
      <displayPosition x="204.0" y="246.0"/>
    </node>
    <node id="Speckle-Filter">
      <displayPosition x="304.0" y="279.0"/>
    </node>
    <node id="Terrain-Correction">
      <displayPosition x="381.0" y="246.0"/>
    </node>
    <node id="BandMerge">
      <displayPosition x="569.0" y="165.0"/>
    </node>
    <node id="BandSelect">
      <displayPosition x="462.0" y="181.0"/>
    </node>
    <node id="LinearToFromdB">
      <displayPosition x="528.0" y="224.0"/>
    </node>
    <node id="Subset">
      <displayPosition x="703.0" y="130.0"/>
    </node>
  </applicationData>
</graph>

.bat file (run on Windows):

@echo off
:: enable delayed expansion - used to resolve variable in loop
:: variable has to be used with '!' instead of '%'
setlocal ENABLEDELAYEDEXPANSION

::::::::::::::::::::::::::::::::::::::::::::
:: User Configuration
::::::::::::::::::::::::::::::::::::::::::::

:: adapt this path to your needs
set gptPath="C:\Program Files\snap\bin\gpt.exe"

::::::::::::::::::::::::::::::::::::::::::::
:: Command line handling
::::::::::::::::::::::::::::::::::::::::::::

:: first parameter is a path to the graph xml
set graphXmlPath=%1

:: second parameter is a path to a parameter file
set parameterFilePath=%2

:: use third parameter for path to source products
set sourceDirectory=%3
:: if sourceDirectory ends with '\' remove it
if %sourceDirectory:~-1%==\ set sourceDirectory=%sourceDirectory:~0,-1%

:: use third parameter for path to target products
set targetDirectory=%4
:: if targetDirectory ends with '\' remove it
if %targetDirectory:~-1%==\ set targetDirectory=%targetDirectory:~0,-1%

:: the fourth parameter is a file prefix for the target product name, 
:: typically indicating the type of processing
set targetFilePrefix=%5

:: Create the target directory
md %targetDirectory%

::::::::::::::::::::::::::::::::::::::::::::
:: Main processing
::::::::::::::::::::::::::::::::::::::::::::

:: double '%' in batch file and only a single '%' on command line
:: '/D' is for directories like Sentinel data. Remove '/D' when you open files.
for /D /R %sourceDirectory% %%F in (S1*.SAFE) do (
  if /I not "%%~xF" == ".SAFE" (
    echo Skipping file %%F as it is not a directory.
  ) else (
    echo.
    :: '~fF' means abolute path of 'F'
    set sourceFile=%%~fF
    echo Processing !sourceFile!
    :: '~nF' means filename without extension of 'F'
    set targetFile=%targetDirectory%\%targetFilePrefix%_%%~nF.dim
    echo targetFile !targetFile!
    set procCmd=%gptPath% %graphXmlPath% -e -p %parameterFilePath% -t "!targetFile!" "!sourceFile!"
    call !procCmd!
  )
)

It is similar as I expected.
The source product files is given as last parameter. This is somehow similar to the usage of -S.
For such a file a Read node is automatically added to the graph.

You graph expects a parameter on the command line which is named sourceFile.

You have two options. Change the command line call in the script to:

set procCmd=%gptPath% %graphXmlPath% -e -p %parameterFilePath% -PsourceFile= "!sourceFile!" -t "!targetFile!"

or remove the read node from the graph and change the Apply-Orbit-File node for the source product to:

<node id="Apply-Orbit-File">
    <operator>Apply-Orbit-File</operator>
    <sources>
       <source>${source}</source>
    </sources>
    <parameters>
...

I have tried both options:

By only changing the command line call, I now get this error:
Error: [NodeId: Read] The 'file' parameter is not set

By only removing the read node and changing the Appy-Orbit-File node I get this error:
Error: Missing source '${source}' in node 'Apply-Orbit-File'

I also tried changing the <source> element of the Apply-Orbit-File node -in the part you suggested I changed- from <source>${sourceFile}</source> to <source>${sourceFile}</source> and get the same error: Error: Missing source '${sourceFile}' in node 'Apply-Orbit-File'

I really appreciate the time and effort you have spent both in making the Bulk Processing tutorial and to answer my queries.

When removing the Read node
Try this variation:

<node id="Apply-Orbit-File">
    <operator>Apply-Orbit-File</operator>
    <sources>
       <source>${sourceProduct}</source>
    </sources>
    <parameters>
...

I don’t see why changing the command line call leads to the “file parameter is not set” error.

Could you add
echo !procCmd!
before the line
call !procCmd!

This will print how the call finally looks like.
Please post the printed line.

this is what is printed:


INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.esa.snap.core.util.EngineVersionCheckActivator: Please check regularly for new updates for the best SNAP experience.
Executing processing graph
INFO: org.esa.s1tbx.commons.io.ImageIOFile: Using FileCacheImageInputStream
INFO: org.hsqldb.persist.Logger: dataFileCache open start
OpenSearch: https://scihub.copernicus.eu/gnss/search?q=platformname:Sentinel-1 AND platformnumber:A AND producttype:AUX_POEORB AND beginposition:[2022-03-01T00:00:000Z TO 2022-03-31T24:00:000Z]
OpenSearch: 31 total results on 1 pages.
OpenSearch: https://scihub.copernicus.eu/gnss/search?q=platformname:Sentinel-1 AND platformnumber:A AND producttype:AUX_POEORB AND beginposition:[2022-03-01T00:00:000Z TO 2022-03-31T24:00:000Z]
 done.
org.esa.snap.core.gpf.graph.GraphException: [NodeId: Terrain-Correction] Operator 'RangeDopplerGeocodingOp': '${resolution}' cannot be converted to a double.
        at org.esa.snap.core.gpf.graph.NodeContext.initTargetProduct(NodeContext.java:79)
        at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:199)
        at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:182)
        at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:182)
        at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:182)
        at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:182)
        at org.esa.snap.core.gpf.graph.GraphContext.initOutput(GraphContext.java:166)
        at org.esa.snap.core.gpf.graph.GraphContext.<init>(GraphContext.java:85)
        at org.esa.snap.core.gpf.graph.GraphContext.<init>(GraphContext.java:58)
        at org.esa.snap.core.gpf.graph.GraphProcessor.executeGraph(GraphProcessor.java:127)
        at org.esa.snap.core.gpf.main.DefaultCommandLineContext.executeGraph(DefaultCommandLineContext.java:86)
        at org.esa.snap.core.gpf.main.CommandLineTool.executeGraph(CommandLineTool.java:547)
        at org.esa.snap.core.gpf.main.CommandLineTool.runGraph(CommandLineTool.java:391)
        at org.esa.snap.core.gpf.main.CommandLineTool.runGraphOrOperator(CommandLineTool.java:287)
        at org.esa.snap.core.gpf.main.CommandLineTool.run(CommandLineTool.java:188)
        at org.esa.snap.core.gpf.main.CommandLineTool.run(CommandLineTool.java:121)
        at org.esa.snap.core.gpf.main.GPT.run(GPT.java:59)
        at org.esa.snap.core.gpf.main.GPT.main(GPT.java:37)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.esa.snap.runtime.Launcher.lambda$run$0(Launcher.java:55)
        at org.esa.snap.runtime.Engine.runClientCode(Engine.java:189)
        at org.esa.snap.runtime.Launcher.run(Launcher.java:51)
        at org.esa.snap.runtime.Launcher.main(Launcher.java:31)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.exe4j.runtime.LauncherEngine.launch(LauncherEngine.java:84)
        at com.exe4j.runtime.WinLauncher.main(WinLauncher.java:94)
        at com.install4j.runtime.launcher.WinLauncher.main(WinLauncher.java:25)
Caused by: org.esa.snap.core.gpf.OperatorException: Operator 'RangeDopplerGeocodingOp': '${resolution}' cannot be converted to a double.
        at org.esa.snap.core.gpf.internal.OperatorContext.injectConfiguration(OperatorContext.java:1149)
        at org.esa.snap.core.gpf.internal.OperatorContext.initializeOperator(OperatorContext.java:526)
        at org.esa.snap.core.gpf.internal.OperatorContext.getTargetProduct(OperatorContext.java:298)
        at org.esa.snap.core.gpf.Operator.getTargetProduct(Operator.java:385)
        at org.esa.snap.core.gpf.graph.NodeContext.initTargetProduct(NodeContext.java:77)
        ... 32 more
Caused by: com.bc.ceres.binding.ConversionException: '${resolution}' cannot be converted to a double.
        at com.bc.ceres.binding.converters.NumberConverter.parse(NumberConverter.java:35)
        at com.bc.ceres.binding.converters.NumberConverter.parse(NumberConverter.java:22)
        at com.bc.ceres.binding.dom.DefaultDomConverter$SingleValueChildConverter.convertDomToValue(DefaultDomConverter.java:399)
        at com.bc.ceres.binding.dom.DefaultDomConverter.convertDomChildToProperty(DefaultDomConverter.java:193)
        at com.bc.ceres.binding.dom.DefaultDomConverter.convertDomChildToPropertySet(DefaultDomConverter.java:182)
        at com.bc.ceres.binding.dom.DefaultDomConverter.convertDomToPropertySet(DefaultDomConverter.java:165)
        at com.bc.ceres.binding.dom.DefaultDomConverter.convertDomToValue(DefaultDomConverter.java:158)
        at org.esa.snap.core.gpf.internal.OperatorContext.configureOperator(OperatorContext.java:1165)
        at org.esa.snap.core.gpf.internal.OperatorContext.injectConfiguration(OperatorContext.java:1145)
        ... 36 more

Error: [NodeId: Terrain-Correction] Operator 'RangeDopplerGeocodingOp': '${resolution}' cannot be converted to a double.

This is already what is printed by SNAP/GPT and not what is printed by the script.

However, this shows that the parameter resolution is not provided in your property file.

Apologies for the late reply,
Indeed I had to define the resolution as you suggested.

Thank you for all your support, the processing now works!