[SNAP][GPT][Sentinel3] subsetting sentinel3 with subset tool using graph processing tool

snap
#1

Dears,
I am trying to create subset of sentinel3 images using the subset command from the graph processing tool: the idea is to extract a portion of an images, keeping all the bands, masks and metadata related to the extracted image portion in the .SEN3 native format.
I would like to know if its possible to keep the native format of the sentinel3 data after subseting:
ie: the data rep (with .SEN3) containing all the .nc and related manifest

I have made a xml file containing the required info, launched a first test
and It works but the output format is completely changed.

I cannot find clear info on how to do this : my input files contains this:
geo_coordinates.nc Oa02_radiance.nc Oa05_radiance.nc Oa08_radiance.nc Oa11_radiance.nc Oa14_radiance.nc Oa17_radiance.nc Oa20_radiance.nc removed_pixels.nc tie_meteo.nc
instrument_data.nc Oa03_radiance.nc Oa06_radiance.nc Oa09_radiance.nc Oa12_radiance.nc Oa15_radiance.nc Oa18_radiance.nc Oa21_radiance.nc tie_geo_coordinates.nc time_coordinates.nc Oa01_radiance.nc Oa04_radiance.nc Oa07_radiance.nc Oa10_radiance.nc Oa13_radiance.nc Oa16_radiance.nc Oa19_radiance.nc qualityFlags.nc tie_geometries.nc xfdumanifest.xml

but my output file contains this:
altitude.hdr FWHM_band_17.img FWHM_band_7.hdr lambda0_band_17.img lambda0_band_7.hdr Oa06_radiance.img Oa17_radiance.hdr solar_flux_band_14.img solar_flux_band_4.hdr
altitude.img FWHM_band_18.hdr FWHM_band_7.img lambda0_band_18.hdr lambda0_band_7.img Oa07_radiance.hdr Oa17_radiance.img solar_flux_band_15.hdr solar_flux_band_4.img
detector_index.hdr FWHM_band_18.img FWHM_band_8.hdr lambda0_band_18.img lambda0_band_8.hdr Oa07_radiance.img Oa18_radiance.hdr solar_flux_band_15.img solar_flux_band_5.hdr
detector_index.img FWHM_band_19.hdr FWHM_band_8.img lambda0_band_19.hdr lambda0_band_8.img Oa08_radiance.hdr Oa18_radiance.img solar_flux_band_16.hdr solar_flux_band_5.img
frame_offset.hdr FWHM_band_19.img FWHM_band_9.hdr lambda0_band_19.img lambda0_band_9.hdr Oa08_radiance.img Oa19_radiance.hdr solar_flux_band_16.img solar_flux_band_6.hdr
frame_offset.img FWHM_band_1.hdr FWHM_band_9.img lambda0_band_1.hdr lambda0_band_9.img Oa09_radiance.hdr Oa19_radiance.img solar_flux_band_17.hdr solar_flux_band_6.img
FWHM_band_10.hdr FWHM_band_1.img lambda0_band_10.hdr lambda0_band_1.img latitude.hdr Oa09_radiance.img Oa20_radiance.hdr solar_flux_band_17.img solar_flux_band_7.hdr
FWHM_band_10.img FWHM_band_20.hdr lambda0_band_10.img lambda0_band_20.hdr latitude.img Oa10_radiance.hdr Oa20_radiance.img solar_flux_band_18.hdr solar_flux_band_7.img
FWHM_band_11.hdr FWHM_band_20.img lambda0_band_11.hdr lambda0_band_20.img longitude.hdr Oa10_radiance.img Oa21_radiance.hdr solar_flux_band_18.img solar_flux_band_8.hdr
FWHM_band_11.img FWHM_band_21.hdr lambda0_band_11.img lambda0_band_21.hdr longitude.img Oa11_radiance.hdr Oa21_radiance.img solar_flux_band_19.hdr solar_flux_band_8.img
FWHM_band_12.hdr FWHM_band_21.img lambda0_band_12.hdr lambda0_band_21.img Oa01_radiance.hdr Oa11_radiance.img quality_flags.hdr solar_flux_band_19.img solar_flux_band_9.hdr
FWHM_band_12.img FWHM_band_2.hdr lambda0_band_12.img lambda0_band_2.hdr Oa01_radiance.img Oa12_radiance.hdr quality_flags.img solar_flux_band_1.hdr solar_flux_band_9.img
FWHM_band_13.hdr FWHM_band_2.img lambda0_band_13.hdr lambda0_band_2.img Oa02_radiance.hdr Oa12_radiance.img solar_flux_band_10.hdr solar_flux_band_1.img tie_point_grids
FWHM_band_13.img FWHM_band_3.hdr lambda0_band_13.img lambda0_band_3.hdr Oa02_radiance.img Oa13_radiance.hdr solar_flux_band_10.img solar_flux_band_20.hdr vector_data
FWHM_band_14.hdr FWHM_band_3.img lambda0_band_14.hdr lambda0_band_3.img Oa03_radiance.hdr Oa13_radiance.img solar_flux_band_11.hdr solar_flux_band_20.img
FWHM_band_14.img FWHM_band_4.hdr lambda0_band_14.img lambda0_band_4.hdr Oa03_radiance.img Oa14_radiance.hdr solar_flux_band_11.img solar_flux_band_21.hdr
FWHM_band_15.hdr FWHM_band_4.img lambda0_band_15.hdr lambda0_band_4.img Oa04_radiance.hdr Oa14_radiance.img solar_flux_band_12.hdr solar_flux_band_21.img
FWHM_band_15.img FWHM_band_5.hdr lambda0_band_15.img lambda0_band_5.hdr Oa04_radiance.img Oa15_radiance.hdr solar_flux_band_12.img solar_flux_band_2.hdr
FWHM_band_16.hdr FWHM_band_5.img lambda0_band_16.hdr lambda0_band_5.img Oa05_radiance.hdr Oa15_radiance.img solar_flux_band_13.hdr solar_flux_band_2.img
FWHM_band_16.img FWHM_band_6.hdr lambda0_band_16.img lambda0_band_6.hdr Oa05_radiance.img Oa16_radiance.hdr solar_flux_band_13.img solar_flux_band_3.hdr
FWHM_band_17.hdr FWHM_band_6.img lambda0_band_17.hdr lambda0_band_6.img Oa06_radiance.hdr Oa16_radiance.img solar_flux_band_14.hdr solar_flux_band_3.img

Could you please provide some advises.
thank you in advance
best regards,
Yoann

#2

No, this unfortunately not possible. Writing to the SEN3 format is not supported.
The default output format is BEAM-DIMAP. This is what you got.
You could switch for example to a single Netcdf file by adding the option ‘-f NetCDF4-CF’ to the command line. But the native SEN3 format is not supported.

#3

Thank you for your answer. Its very unfortunate indeed. Is this something that is in the pipeline? or this is not something that is being considered at all?

#4

At the moment this is not planned.

#5

OK, thank you for the quick answer.
best regards,

#6

I have just realized that the resulting subset is greater than the subset defined in the xml by the geocoordinate of the bounding box?
Is it expected ? it is supposed to be a rectangle and I see that the subset is a nearly perfect square…is it something I am doing wrong?

Update : in fact I do not get any error, the processing is going as expected (i guess), but the result is not a subset but the input data to its full extent… here are the logs

INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Incompatible GDAL 1.11.4 found on system. Internal GDAL 3.0.0 from distribution will be used.
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Internal GDAL 3.0.0 set to be used by SNAP.
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
INFO: org.esa.snap.core.util.EngineVersionCheckActivator: Please check regularly for new updates for the best SNAP experience.
Executing processing graph
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Internal GDAL 3.0.0 set to be used by SNAP.
INFO: org.hsqldb.persist.Logger: dataFileCache open start
.7299 [main] INFO serverStartup - Nc4Iosp: NetCDF-4 C library loaded (jna_path=’/mount/dmz30/apps/shared-mode/s3mpc/s3operator/.snap/auxdata/netcdf_natives/8.0.8/amd64’, libname=‘netcdf’).
7303 [main] INFO serverStartup - NetcdfLoader: set log level: old=0 new=0
7304 [main] INFO serverStartup - Nc4Iosp: set log level: old=0 new=0
…10%…21%…32%…43%…54%…65%…75%…86%… done.

#9 
<graph id="ROI_test">
  <version>1.0</version>
  <node id="extract_roi">
    <operator>Subset</operator>
    <sources>
      <source>${source}</source>
    </sources>
    <parameters class="com.bc.ceres.binding.dom.XppDomElement">
      <sourceBands/>
      <referenceBand/>
      <geoRegion>'POLYGONE((119.50000 31.000000, 124.00000 31.000000, 124.00000 37
.000000, 119.50000 37.000000, 119.50000 31.000000))'</geoRegion>
      <subSamplingX>1</subSamplingX>
      <subSamplingY>1</subSamplingY>
      <fullSwath>False</fullSwath>
      <tiePointGridNames/>
      <copyMetadata>true</copyMetadata>
    </parameters>
  </node>
</graph>
#10

here is my xml file and how I launch the command:
./gpt ROI_SNAP/ROI_lib/ROI_test1.xml -Ssource=ROI_SNAP/ROI_input/S3B_OL_1_EFR____20211108T015646_20211108T015946_20211108T034615_0179_059_060_2340_LN1_O_NR_002.SEN3/xfdumanifest.xml -t /ROI_SNAP/ROI_input/S3B_OL_1_EFR____20211108T015646_20211108T015946_20211108T034615_0179_059_060_2340_LN1_O_NR_002_ROI3.SEN3/ -f NetCDF4-CF

#11

typo?: POLYGON, not POLYGONE.

#12

The subset is always rectangular and represents the outer bounding box of the provided polygon. That’s why it can be bigger.
Maybe you reproject the data to WGS84 beforehand, then the coordinates would form rectangle on the grid. On the original satellite grid, the polygon you have provided is rectangular but rotated.
This is shown in the following image.

image
image1367×854 165 KB

@gnwiii Surprisingly it works also with ‘Polygone’ because the code checks if the text starts with ‘Polygon’ and then moves on to the ‘((’.

#13

SNAP_error
SNAP_error1037×925 213 KB

Good morning. I understand, However, if we look at the provided image with the polygon overlaid on top of the resuting subset, it seems obvious that the subset did not work as anticipated. Since no error is returned by the script i struggle to see where the mistake is. I have corrected the Typo (thank you @gnwiii), but no change.

#14

I was able to reproduce your result. The reason is that the geometry is ignored, because of the single quotes.

The graph should work if it is like this:

<graph id="ROI_test">
  <version>1.0</version>
  <node id="extract_roi">
    <operator>Subset</operator>
    <sources>
      <source>${source}</source>
    </sources>
    <parameters class="com.bc.ceres.binding.dom.XppDomElement">
      <sourceBands/>
      <referenceBand/>
      <geoRegion>POLYGON((119.50000 31.000000, 124.00000 31.000000, 124.00000 37
.000000, 119.50000 37.000000, 119.50000 31.000000))</geoRegion>
      <subSamplingX>1</subSamplingX>
      <subSamplingY>1</subSamplingY>
      <fullSwath>False</fullSwath>
      <tiePointGridNames/>
      <copyMetadata>true</copyMetadata>
    </parameters>
  </node>
</graph>

My result was in this case like:

image
image1834×1335 156 KB

So, the problem is that no error is reported.
[SNAP-1485] Provided geoRegion is not considered in subsetting graph if quoted - JIRA (atlassian.net)

1 Like
#15

Wonderful, it worked, thank you very much for helping.
I wish you a good day.

1 Like