Processing S3 OLCI Level 1 EFR Images - SNAP Error
I am trying to process some S3 OLCI Level 1 EFR images, for example:
S3A_OL_1_EFR____20220624T090255_20220624T090555_20220625T133304_0179_086_378_1980_MAR_O_NT_002.SEN3
I am using the current version of SNAP 11.0.0 (24.10.2024 13:00 UTC) on an Apple M3 Max with 48 GB RAM (macOS Sonoma v14.1).
The processing chain involves only a Subset followed by Idepix, and I have installed the Idepix.Olci plugin.
Processing Graph
<graph id="Graph">
<version>1.0</version>
<node id="Subset">
<operator>Subset</operator>
<sources>
<sourceProduct>${sourceProduct}</sourceProduct>
</sources>
<parameters class="com.bc.ceres.binding.dom.XppDomElement">
<sourceBands/>
<geoRegion>POLYGON ((20 58, 21 58, 21 59, 20 59, 20 58))</geoRegion>
<subSamplingX>1</subSamplingX>
<subSamplingY>1</subSamplingY>
<fullSwath>false</fullSwath>
<tiePointGridNames/>
<copyMetadata>true</copyMetadata>
</parameters>
</node>
<node id="Idepix">
<operator>Idepix.Olci</operator>
<sources>
<sourceProduct refid="Subset"/>
</sources>
<parameters class="com.bc.ceres.binding.dom.XppDomElement">
<computeCloudBuffer>true</computeCloudBuffer>
<cloudBufferWidth>2</cloudBufferWidth>
</parameters>
</node>
</graph>
Python Code
import os
import subprocess
gpt = "/Applications/esa-snap/bin/gpt"
arg_graph = "graph.xml"
in_folder = "products/"
out_folder = "outputs/"
# List all files matching the pattern
all_img = [
os.path.join(dp, f) for dp, dn, filenames in os.walk(in_folder)
for f in filenames if f.endswith("xfdumanifest.xml")
]
n_img = all_img[:1]
# Loop through each file
for i, img_path in enumerate(n_img, start=1):
print(f"--------------- {i} / {len(n_img)} ----------------")
# Extract the input file name and construct output name
base_name = os.path.basename(os.path.dirname(img_path)) # Extract folder name without path
out_name = f"{base_name}.Idepix.nc"
subset_out = os.path.join(out_folder, out_name)
# Construct the argument string for GPT
idepix_arg = f"{arg_graph} -SsourceProduct={img_path} -t {subset_out} -f NetCDF4-BEAM"
# Execute the GPT command
subprocess.run([gpt] + idepix_arg.split(), check=True)
Error Message
I encounter the following error message when running the above code:
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.esa.snap.runtime.Engine (file:/Applications/esa-snap/snap/modules/ext/org.esa.snap.snap-core/org-esa-snap/snap-runtime.jar) to method java.lang.ClassLoader.initializePath(java.lang.String)
WARNING: Please consider reporting this to the maintainers of org.esa.snap.runtime.Engine
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
WARNING: org.esa.snap.core.util.ServiceLoader: org.esa.snap.core.gpf.OperatorSpi: Provider eu.esa.opt.meris.sdr.aerosol.AerosolMergerOp$Spi not found
WARNING: org.esa.snap.core.util.ServiceLoader: org.esa.snap.core.gpf.OperatorSpi: Provider eu.esa.opt.meris.sdr.aerosol.ModisAerosolOp$Spi not found
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.esa.snap.core.util.EngineVersionCheckActivator: Please check regularly for new updates for the best SNAP experience.
Executing processing graph
100% done.
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGILL (0x4) at pc=0x0000000185b27ba4, pid=8410, tid=11011
#
# JRE version: OpenJDK Runtime Environment (11.0.19+7) (build 11.0.19+7-LTS)
# Java VM: OpenJDK 64-Bit Server VM (11.0.19+7-LTS, mixed mode, tiered, g1 gc, bsd-amd64)
# Problematic frame:
# C [libtensorflow_framework.1.dylib+0x7ba4] tensorflow::monitoring::MetricDef<(tensorflow::monitoring::MetricKind)1, long long, 2>::MetricDef<char [11], char [7]>(absl::string_view, absl::string_view, char const (&) [11], char const (&) [7])+0x44
#
# No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /Users/S3_OLCI_Processing/learn-olci/2_OLCI_advanced/hs_err_pid8410.log
#
# If you would like to submit a bug report, please visit:
# https://bell-sw.com/support
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
However, when I run the same graph substituting the Idepix node for C2RCC, the code runs fine and the image is processed correctly.
<graph id="Graph">
<version>1.0</version>
<node id="Subset">
<operator>Subset</operator>
<sources>
<sourceProduct>${sourceProduct}</sourceProduct>
</sources>
<parameters class="com.bc.ceres.binding.dom.XppDomElement">
<sourceBands/>
<geoRegion>POLYGON ((20 58, 21 58, 21 59, 20 59, 20 58))</geoRegion>
<subSamplingX>1</subSamplingX>
<subSamplingY>1</subSamplingY>
<fullSwath>false</fullSwath>
<tiePointGridNames/>
<copyMetadata>true</copyMetadata>
</parameters>
</node>
<node id="C2rcc">
<operator>c2rcc.olci</operator>
<sources>
<sourceProduct refid="Subset"/>
</sources>
<parameters class="com.bc.ceres.binding.dom.XppDomElement">
<outputAsRrs>true</outputAsRrs>
<thresholdRtosaOOS>0.05</thresholdRtosaOOS>
<validPixelExpression>!quality_flags.invalid AND !quality_flags.land</validPixelExpression>
</parameters>
</node>
</graph>
I have also tried this using SNAP versions 9 and 10 and also on the GUI, however, the same problem occurs.
Any help on why this is happening and how I can process my images would be great!
Thanks