Idepix OLCI with subset

I am trying to process S3A Level 1 images on SNAP 11. The process involves subset and then Idepix.olci, previously this has worked fine on SNAP V9. I have just updated SNAP to the current version and when I run my code, I receive this error:

Error: [NodeId: Idepix.Olci] Product 'Slope-Aspect-Orientation' does not contain a raster with name 'OZA_interp'

or

Error: [NodeId: Idepix.Olci] Selected cloud screening algorithm cannot be used with given input product. Supported sensors are: MERIS, SPOT VGT, MODIS, Landsat-8, SeaWiFS, Sentinel-2 MSI, Sentinel-3 OLCI, PROBA-V, VIIRS. org.esa.snap.core.gpf.graph.GraphException: [NodeId: Idepix.Olci] Selected cloud screening algorithm cannot be used with given input product.

The same issue occurs when I try and run this on using the GUI even when i subset all of the tie-point grids.

The OZA files are in the Tie-Point Grids folder both for the full scene and after I subset the scene (without Idepix).

When I run Idepix.Olci on the full image first then subset it works fine so I think the subset process is not retaining the ‘OZA_interp’. Does anyone know how I can subset the image first and then Idepix without losing the OZA_interp raster?

Example image: S3A_OL_1_EFR____20190714T104001_20190714T104301_20190715T145430_0179_047_051_1980_LN1_O_NT_002.SEN3

Graph subset > idepix (Not working):

temp_20190714_Region3986_not_working.xml (2.2 KB)

Graph idepix > subset (working)

temp_20190714_Region3986_working.xml (2.2 KB)

@dolaf could you have a look?
Thank you

Hi,

the reason for this problem is that the signature for the product type for subsets has slightly changed from SNAP 9 to 10/11, and this has not been properly adapted in Idepix OLCI.

We will fix this and provide the fix in a module update for SNAP 11 or, latest, with the next version SNAP 12.

For the moment, in case you are not doing ‘mass processing’ and just have a reasonable number of input subsets to process, you could change the product type manually in SNAP Desktop to get it running:

  • select the subset product in Product Explorer
  • right-click, then right-click ‘Properties’
  • In the Product Node Properties window, edit the field ‘Product Type’ with the three dots on the right
  • if you change ‘Derived from (OL_1_EFR)’ to ‘Derived from OL_1_EFR’ (just remove the brackets), Idepix should work

In general, please note that for any preprocessing or L2 processor the functionality may be limited or the processing might fail if you use just subsets of the originally supported input products. Subsets may look very different (spatial or band subsets, metadata included or not, etc.), thus essential information might be missing.

Cheers,
Olaf

Hi Olaf,

This update would be really appreciated. This is preventing us from upgrading SNAP v9 in our automated pipelines.

Thanks for your work!

Cheers

James

We had the same issue and I created a little python script that opens the DIMAP files and changes the product type. Just in case you have a lot of files you want to process:

import xml.etree.ElementTree as ET
import os

def modify_product_type(xml_file):
    """
    Reads an XML file, changes the <PRODUCT_TYPE> string to 'OL_1_EFR',
    and saves the modified XML back to the file.

    Args:
        xml_file (str): The path to the XML file.
    """
    try:
        tree = ET.parse(xml_file)
        root = tree.getroot()

        for production_element in root.findall(".//Production"):
            product_type_element = production_element.find("PRODUCT_TYPE")
            if product_type_element is not None:
                product_type_element.text = "OL_1_EFR"
                break
        else:
            print(f"Warning: <PRODUCT_TYPE> element not found in {xml_file}")

        tree.write(xml_file, encoding="ISO-8859-1", xml_declaration=True)
        print(f"Successfully modified and saved: {xml_file}")

    except ET.ParseError:
        print(f"Error: Could not parse {xml_file}. It may not be a valid XML file.")
    except FileNotFoundError:
        print(f"Error: File not found: {xml_file}")
    except Exception as e:
        print(f"An unexpected error occurred: {e}")

def process_xml_files_in_directory(directory):
    """
    Searches for all XML files in the specified directory and applies the
    modify_product_type function to each file.

    Args:
        directory (str): The path to the directory containing XML files.
    """
    try:
        for filename in os.listdir(directory):
            if filename.endswith(".dim"):
                file_path = os.path.join(directory, filename)
                modify_product_type(file_path)
    except FileNotFoundError:
        print(f"Error: Directory not found: {directory}")
    except Exception as e:
        print(f"An unexpected error occurred: {e}")

# Example usage:
directory_path = "your_directory_path"  # Replace with the actual directory path
process_xml_files_in_directory(directory_path)