Process killed when writing to NetCDF4-CF

snap
1

Hello,

We are using Polymer for atmospheric correction of S3 OLCI Level 1 EFR images, followed by mosaicking the l2 files using gpt. We are trying to save the output as NetCDF4-CF format, but the process is being killed. If I try to save in a different format, such as GeoTIFF, it is able to finish processing. I’ve found some other similar issues mentioned on the forum (example1, example2), but no solutions. It seems like memory is running out and killing the process when saving to NetCDF, but I feel as though it shouldn’t be as we have 96G of RAM. Any ideas on what the issue may be and how to fix it?

My xlm is:
mosaic_polymer_LIS.xml (2.7 KB)

My system: macOS 14.5
SNAP version: 11.0
OLCI data LEVEL: S3A_OL_1_EFR

Example:

Running this gpt call

/esa-snap/bin/gpt /code/polymer-4.17beta2/mosaic_polymer_LIS.xml /l2_polymer/S3A_OL_1_EFR____20230809T145335_20230809T145635_20230810T153435_0179_102_082_2160_PS1_O_NT_003_x_polymer_L2.nc /l2_polymer/S3A_OL_1_EFR____20230809T145635_20230809T145935_20230810T153435_0179_102_082_2340_PS1_O_NT_003_x_polymer_L2.nc -t /Daily/test -f NETCDF4-CF

Results in the following where the process is killed:

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.esa.snap.runtime.Engine (file:/esa-snap/snap/modules/ext/org.esa.snap.snap-core/org-esa-snap/snap-runtime.jar) to method java.lang.ClassLoader.initializePath(java.lang.String)
WARNING: Please consider reporting this to the maintainers of org.esa.snap.runtime.Engine
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.esa.snap.core.util.EngineVersionCheckActivator: Please check regularly for new updates for the best SNAP experience.
Executing processing graph
WARNING: org.esa.snap.dataio.netcdf.util.MetadataUtils: Missing configuration property ‘snap.dataio.netcdf.metadataElementLimit’. Using default (100).
WARNING: org.esa.snap.dataio.netcdf.util.MetadataUtils: Missing configuration property ‘snap.dataio.netcdf.metadataElementLimit’. Using default (100).

100% done.

100% done.
15%30%45%…55%…Killed: 9

-Kendra

2

Dear Kendra,

we cannot reproduce this behaviour, unfortunately.

If the process is killed by the operating system - as “Killed 9” suggests - there should be a file called “hs_err_pid …” which contains the crash dump information of the Java runtime. If possible, can you please make this available to us.

You can adjust the amount of memory that SNAP uses in the configuration file
snap-install-dir/etc/snap.conf

default_options="–branding snap --locale en_GB -J-XX:+AggressiveOpts -J-Xverify:none -J-Xms256M -J-Xmx22G

The Xmx sets the upper limit of the RAM that the Java VM is allowed to allocate.

Thanks a lot and sorry for the inconvenience.

Tom

3

Hi Tom,

Thanks for the response! I tried adjusting the amount of memory in the snap.conf file using Xmx, but still got the same error.

Here is the error log:
java-2024-12-04-125952.txt (122.7 KB)

-Kendra

4

Dear Kendra,

thanks for the logfile - this gives some insights:

/jre.bundle/Contents/Home/lib/libawt_lwawt.dylib
“type”:“EXC_BAD_ACCESS”,“signal”:“SIGKILL (Code Signature Invalid)”

It looks like the JDK installation on your system is corrupt, the crash is triggered by a JAVA dynamic library - which we cannot control from within SNAP.

We have two suggestions for you:
a) re-install SNAP
b) inject a different JDK into SNAP

The first is a simple one, which might already help

The latter can be achieved by

  1. download an appropriate JDK version 11 (e.g. OpenJDK) for your CPU architecture and OS
  2. extract that to somewhere on your system
  3. edit <SNAP_INSTALL_DIR>/etc/snap.conf
  4. replace the JDK Home parameter
    jdkhome=“./jre” → jdkhome=“C:/Projects/SNAP/JDK/liberica_11/jdk-11.0.18”
    of course - using the path of your system and not mine :slight_smile:
  5. save

Hope this helps!

Tom