Binning operator fails in SNAP9 for MERIS and MODISA

snap
#1

We have been successfully doing L3bin (Binning) processing for various sensors (SeaWiFS, MERIS, MODIS, VIIRS, OLCI) starting from BEAM until SNAP6.

After upgrading to SNAP9, the processing chains for some sensors, namely MERIS and MODIS, stopped working when using the same binning configuration files created with SNAP6; for other sensors the processing works all right. The errors for MERIS and MODIS look very different, I’ll skip MODIS for now. For MERIS the error looks like all input half-orbits (POLYMER processed L2 files) are rejected due to geometric problems. The command:

gpt -e -Dsnap.pixelGeoCoding.fractionAccuracy=true mod_xml_file.xml

Essential error output:

WARNING: org.esa.snap.binning.operator.BinningOp: Filtered out result of graph ‘/data/carrack1/scratch/projects-nobackup/OCCCI/v6.0-production/tests/workspaces/PB_2010_11_27/merge_ENV_ME_1_RRG____20101127T165441_20101127T173826_________________2624_097_012______DSI_R_NT____.idepix.xml’
WARNING: org.esa.snap.binning.operator.BinningOp: reason: not accepting product ‘ENV_ME_1_RRG____20101127T165441_20101127T173826_________________2624_097_012______DSI_R_NT____.L2_poly’: topLeftMembership=PREVIOUS_PERIODS, topRightMembership=PREVIOUS_PERIODS, bottomLeftMembership=PREVIOUS_PERIODS, bottomRightMembership=PREVIOUS_PERIODS, startTime=27-NOV-2010 16:54:41.000000, endTime=27-NOV-2010 17:38:26.000000
WARNING: org.esa.snap.dataio.netcdf.util.MetadataUtils: Missing configuration property ‘snap.dataio.netcdf.metadataElementLimit’. Using default (100).

As the result of the “warnings” above, binner does not detect any valid input and generates error output:

Error: [NodeId: binningNode] No valid input products found during spatial binning. Quitting operator

I remind: this command with the same input perfectly works in SNAP6 environment. What has been changed in Binning operator between SNAP versions 6 and 9?

To make things even more confused, if using SNAP8 same way, we have only one sensor failing: MERIS. For MODIS this still works in SNAP8 (not in 9).

Please let us know what should we change in processing setup to make SNAP9 working with MERIS and MODIS too. Another hint: our colleagues using similar to Binning operator LakeAggregate (a specific user customized operator, probably not suitable for us) do not have such problems with MERIS and MODIS in SNAP9 environment.

#2

There seems to be a dirty fix preventing the process from crashing and even providing results looking reasonable, I just found it accidently. But I do not like it. One needs to comment out two lines in the binning congig file, shown commented here:

                 </postProcessor>
                 <!--timeFilterMethod>SPATIOTEMPORAL_DATA_DAY</timeFilterMethod>
                 <minDataHour>10</minDataHour-->
                 <periodDuration>1</periodDuration> 
                 <superSampling>3</superSampling>
                 <outputType>Product</outputType>
                 <outputFormat>NetCDF4-BEAM</outputFormat>
                 <outputFile>MER_20101127_Rrs_L3.dim</outputFile>
              </parameters>

I know these commented parameters are important, at least in older SNAP versions, and minDataHour differ from sensor to sensor.

I would ge grateful for somebody to explain what is going on with this binning.

Also the problem of failing binning for MODIS is still there, it is not related to acceptance/rejection of orbits.

#3

@TomBlock can you have a look? Thanks

#4

Thank you, Diana!

Hi Tom,

Here is Andrei from PML, we worked together on OC-CCI project. I think I see what is going on with SNAP9 binning for MERIS, and I think this is a bug which was introduced probably in SNAP8. If I do NOT comment out the two lines in the config file above, time filter rejects every L2 input file - ‘SPATIOTEMPORAL_DATA_DAY’ could not be activated. If I comment out as shown, default filtering method works: ‘TIME_RANGE’, and processing works but L3bin results differ from ones obtained with SNAP6; the latter are correct, results of SNAP9 are not.

I think MODIS binning also may fail with the same problem (now it fails for us at a previous stage, but I suspect when/if fixed, it would also fail on timeFilterMethod). With SNAP6 all 6 OC-CCI sensors have worked OK, L3bin part.

Cheers,
Andrei