IW3 missing when downloading product from Data Space

Hi all

I am experiencing a weird issue when downloading products from the new Data Space portal. Once downloaded on my local computer, I’m trying to perform a TOPSAR Split operation but it seems the IW3 subswath is outside the raster boundaries (see captures). I’m using SNAP 9.0.0 and the issue can be reproduced using S1A_IW_SLC__1SDV_20230731T053520_20230731T053547_049663_05F8D1_C3A6. But as far as I investigate, the problem arises with any SLC IW products.

Capture d’écran du 2023-08-01 09-16-18

Interestingly, everything seems to work ok when I download a product from the old Scihub portal. The IW3 is part of the product raster and the TOPSAR Split operation is successful.

Capture d’écran du 2023-08-01 09-17-08

Looking at product metadata, I noticed raster dimension from Data Space and Scihub is also different:

Scihub

num_output_lines = 13725
num_samples_per_line = 75436

Data Space

num_output_lines = 13671
num_samples_per_line = 49735

Which I guess is consistent with the Split issue described above.

Please also note, I don’t modify the product at any stage, I ran this test on freshly downloaded, unedited products.
Does it sound familiar to somebody ? Thank you very much for your insights

Julien

After further investigation, it appears the issue shows up when SNAP directly opens the zipped product. When I unzip the product myself and open the unzipped product with SNAP, IW3 become visible as expected.

Maybe it relates to some built-in unzipping strategy ?

Hello @JulienPiquot

Thank you for reporting this issue. A JIRA ticket SNAP-3537 has been created.

1 Like

We’re also facing this issue. I went through the code with the debugger and javax.imageio.ImageIO encounters the following exception when trying to read the IW3 file: java.util.zip.ZipException: invalid LOC header (bad signature)

Stack trace:

java.util.zip.ZipException: invalid LOC header (bad signature)
	at java.util.zip.ZipFile.read(Native Method)
	at java.util.zip.ZipFile.access$1400(ZipFile.java:61)
	at java.util.zip.ZipFile$ZipFileInputStream.read(ZipFile.java:741)
	at javax.imageio.stream.FileCacheImageInputStream.readUntil(FileCacheImageInputStream.java:142)
	at javax.imageio.stream.FileCacheImageInputStream.read(FileCacheImageInputStream.java:187)
	at javax.imageio.stream.ImageInputStreamImpl.readFully(ImageInputStreamImpl.java:351)
	at javax.imageio.stream.ImageInputStreamImpl.readFully(ImageInputStreamImpl.java:361)
	at com.sun.media.imageioimpl.plugins.pnm.PNMImageReaderSpi.canDecodeInput(Unknown Source)
	at javax.imageio.ImageIO$CanDecodeInputFilter.filter(ImageIO.java:567)
	at javax.imageio.spi.FilterIterator.advance(ServiceRegistry.java:834)
	at javax.imageio.spi.FilterIterator.<init>(ServiceRegistry.java:828)
	at javax.imageio.spi.ServiceRegistry.getServiceProviders(ServiceRegistry.java:519)
	at javax.imageio.ImageIO.getImageReaders(ImageIO.java:646)
	at org.esa.s1tbx.io.geotiffxml.GeoTiffUtils.getTiffIIOReader(GeoTiffUtils.java:15)
	at org.esa.s1tbx.io.sentinel1.Sentinel1Level1Directory.addImageFile(Sentinel1Level1Directory.java:90)
	at org.esa.s1tbx.commons.io.AbstractProductDirectory.findImages(AbstractProductDirectory.java:141)
	at org.esa.s1tbx.commons.io.AbstractProductDirectory.findImages(AbstractProductDirectory.java:168)
	at org.esa.s1tbx.io.sentinel1.Sentinel1Level1Directory.createProduct(Sentinel1Level1Directory.java:987)
	at org.esa.s1tbx.io.sentinel1.Sentinel1ProductReader.readProductNodesImpl(Sentinel1ProductReader.java:111)
	at org.esa.snap.core.dataio.AbstractProductReader.readProductNodes(AbstractProductReader.java:178)
	at org.esa.snap.core.gpf.common.ReadOp.initialize(ReadOp.java:168)
	at org.esa.snap.core.gpf.internal.OperatorContext.initializeOperator(OperatorContext.java:528)
	at org.esa.snap.core.gpf.internal.OperatorContext.getTargetProduct(OperatorContext.java:298)
	at org.esa.snap.core.gpf.Operator.getTargetProduct(Operator.java:385)
	at org.esa.snap.core.gpf.graph.NodeContext.initTargetProduct(NodeContext.java:77)
	at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:199)
	at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:182)
	at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:182)
	at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:182)
	at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:182)
	at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:182)
	at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:182)
	at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:182)
	at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:182)
	at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:182)
	at org.esa.snap.core.gpf.graph.GraphContext.initOutput(GraphContext.java:166)
	at org.esa.snap.core.gpf.graph.GraphContext.<init>(GraphContext.java:85)
	at org.esa.snap.core.gpf.graph.GraphContext.<init>(GraphContext.java:58)
	at org.esa.snap.core.gpf.graph.GraphProcessor.executeGraph(GraphProcessor.java:118)

I’ll also log a ticket on the Copernicus Data Space Ecosystem, because this might be something they have to fix on their end.

1 Like

Yes, please always do this so that they know about the issue.

FYI, I reported the problem to the Copernicus Data Space Ecosystem on 30/08/2023.
I did some further investigation and provided came up with a minimal code snippet to reproduce the error in Java 1.8.

package org.example;

import java.io.IOException;
import java.io.InputStream;
import java.util.Enumeration;
import java.util.zip.ZipEntry;
import java.util.zip.ZipException;
import java.util.zip.ZipFile;

public class Main {
    public static void main(String[] args) {
        String zipPath = "/home/stijn/Downloads/S1A_IW_SLC__1SDV_20230829T054319_20230829T054347_050086_0606E8_2748.CDSE.zip";

        try (ZipFile zipFile = new ZipFile(zipPath)) {
            Enumeration<? extends ZipEntry> entries = zipFile.entries();
            while (entries.hasMoreElements()) {
                ZipEntry entry = entries.nextElement();
                System.out.println(entry.getName());
                if (!entry.isDirectory()) {
                    try (InputStream inputStream = zipFile.getInputStream(entry)) {
                        inputStream.read();
                    } catch (ZipException e) {
                        System.err.println("Failed to read " + entry.getName());
                    }
                }
            }
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }
}

Output:

Failed to read S1A_IW_SLC__1SDV_20230829T054319_20230829T054347_050086_0606E8_2748.SAFE/measurement/s1a-iw3-slc-vh-20230829t054321-20230829t054346-050086-0606e8-003.tiff
Failed to read S1A_IW_SLC__1SDV_20230829T054319_20230829T054347_050086_0606E8_2748.SAFE/measurement/s1a-iw3-slc-vv-20230829t054321-20230829t054346-050086-0606e8-006.tiff
Failed to read S1A_IW_SLC__1SDV_20230829T054319_20230829T054347_050086_0606E8_2748.SAFE/preview/icons/logo.png
Failed to read S1A_IW_SLC__1SDV_20230829T054319_20230829T054347_050086_0606E8_2748.SAFE/preview/map-overlay.kml
Failed to read S1A_IW_SLC__1SDV_20230829T054319_20230829T054347_050086_0606E8_2748.SAFE/preview/product-preview.html
Failed to read S1A_IW_SLC__1SDV_20230829T054319_20230829T054347_050086_0606E8_2748.SAFE/preview/quick-look.png
Failed to read S1A_IW_SLC__1SDV_20230829T054319_20230829T054347_050086_0606E8_2748.SAFE/preview/thumbnail.png
Failed to read S1A_IW_SLC__1SDV_20230829T054319_20230829T054347_050086_0606E8_2748.SAFE/support/s1-level-1-calibration.xsd
Failed to read S1A_IW_SLC__1SDV_20230829T054319_20230829T054347_050086_0606E8_2748.SAFE/support/s1-level-1-measurement.xsd
Failed to read S1A_IW_SLC__1SDV_20230829T054319_20230829T054347_050086_0606E8_2748.SAFE/support/s1-level-1-noise.xsd
Failed to read S1A_IW_SLC__1SDV_20230829T054319_20230829T054347_050086_0606E8_2748.SAFE/support/s1-level-1-product.xsd
Failed to read S1A_IW_SLC__1SDV_20230829T054319_20230829T054347_050086_0606E8_2748.SAFE/support/s1-level-1-quicklook.xsd
Failed to read S1A_IW_SLC__1SDV_20230829T054319_20230829T054347_050086_0606E8_2748.SAFE/support/s1-level-1-rfi.xsd
Failed to read S1A_IW_SLC__1SDV_20230829T054319_20230829T054347_050086_0606E8_2748.SAFE/support/s1-map-overlay.xsd
Failed to read S1A_IW_SLC__1SDV_20230829T054319_20230829T054347_050086_0606E8_2748.SAFE/support/s1-object-types.xsd
Failed to read S1A_IW_SLC__1SDV_20230829T054319_20230829T054347_050086_0606E8_2748.SAFE/support/s1-product-preview.xsd

I provided my findings on 06/09/2023, but the problem remains unsolved.

Since it is a compatibility issue with the built-in ZIP reader in Java 1.8, I’m afraid this cannot be solved easily on the side of SNAP either.

1 Like

There is a bug in JDK that might be related:
https://bugs.openjdk.org/browse/JDK-8298530

Good day Ma’am/Sir:

I have encountered the same problem with the one you have here. May I ask the full details on how you resolved this problem, or did you retrieve the IW3? Thank you and Godspeed!

@KVSCodaste
you can unzip the file and utilize the IW3.

With regards to your reply sir, may I ask how I can show IW3 on my Snap software?

Well, if I have understood your problem correctly, You can import ‘manifest.safe’ file into snap after unzipping the downloaded Sentinel-1 Product.

1 Like

But, its not to process calibration… Why? How to solve it?

I noticed that if a compression level other than 0 is given to the data, the software reads it correctly

This problem is solved in v.10 of SNAP! We are in the last phase of tests and SNAP 10 shall be ready to release very soon

1 Like

Dear Magdalena,

Thank You very much for the fast reply,

This problem also introduces critical issues in the processing phase when the data is not read correctly, therefore a quick solution would be necessary

Furthermore, a level 0 data compression results in double use of disk space, from 4GB to 8GB per single file

This worked for me!