When I try to save a band set (B2,B3,B4,B8) of Sentinel-2 image as GeoTiff-BigTIFF, I get an error message “Product writer is unable to write this product as ‘GeoTIFF-BigTIFF’: Cannot write multisize products. Consider resampling the product first.” When I save as BEAM-DIMAP, I get a set of 4 ENVI images that are perfectly the same size (and the same geographic location). Very similar errors are produced also in graph builder when trying to limit the scope to bands of a certain resolution.
Hi Yrjö, I think that TIFF does not support bands with differing pixel sizes - therefore you need to resample the bands into a common resolution first if you want to stick them into a single TIFF.
Bands B2,B3,B4,B8 are all 10-m bands - no differing pixel sizes. This is really a bug in the way SNAP handles Sentinel-2 images. If some obscure ghost components (e.g. cloud masks or detector masks) are of different sizes, they should not affect the output of BandSelect operator.
I know that Resample operator can be used as a way around this bug. Unfortunately this alternative puts a heavy penalty on elapsed time, CPU time, and memory footprint. The situation is worsened in graphs, where the Resample operator seems to be needed practically after every operation you apply to the selected, same-size bands. Because the Resample operator is just a dummy nuisance step imposed by the SNAP architecture/implementation (the resulting image data is bit-to-bit identical to that what I requested - and what I got in the BEAM-DIMAP alternative), the need for this type of resource-consuming null-operators should be eliminated. For instance in the F-TEP case, we have a tile of Sentinel-2 data at 10-m resolution. Adding two Resample operations in a re-processing graph - just as a way around some SNAP artefacts - puts the end user waiting tens of minutes more.
actually you need the resampling only once at the beginning of your graph and the output can be used in the successive steps.
And if you only deal with in your chain with the 10m bands it adds no, or at least only a little extra processing time, because, as you already said, it is a no-op.
I tried the use case you have described in different ways and in 5.0.x and 6 BETA versions.
In the Graph Builder, on the command line and doing the steps separately in the GUI. And it works.
It is not slower then simply removing all bands and masks from the product. This can be easiest done, by unchecking from the context menu of the product the option “Group Nodes by Type”. Afterwards you can select all the nodes you don’t want and hit DEL-Key. Then you can export the product with out resampling. It takes the same time as when doing Resampling and BandSelect in the Graph Builder.
What slows done the export is the format. Big GeoTiff is slower.
For my scene it took:
DIMAP: 76 seconds
Big GeoTiff: 4 minutes
GeoTiff: 80 seconds
If you only want to have these 4 bands you have mentioned, you can use the ‘normal’ GeoTiff format, because it doesn’t exceed the 2GB barrier.
I’ve attached my graph which I have used on the command line with gpt. S2_Resample_BandSelect.xml (1.5 KB)
Thank you for your proposal to get around this bug. Your graph seems to do what it is supposed to do. My original aim (in a graph made by my colleagues) was to select bands and make a spatial subset over a bounding box of a polygon (using the Subset operator). Can you make a graph demonstrating this without including the Resample operator twice ?
I tried your second script with my dataset. The BEAM-DIMAP version went smoothly. When I switched to GeoTIFF-BigTIFF format I got again the error message:
“Error: [NodeId: Write] Product writer is unable to write this product as ‘GeoTIFF-BigTIFF’: Cannot write multisize products. Consider resampling the product first.”
You’re right. Probably I mixed something up while testing on SNAP 5.x.
But on SNAP 6 BETA it is working. Is it an option for you to try the current preview?
I come back to testing later, but naturally I have gpt on PATH. The SNAP installation script creates a link to /usr/local/bin.
tteyar@espvm30301:~/red/prog/fire/dun$ gpt --diag
SEVERE: org.esa.s2tbx.dataio.gdal.activator.GDALDistributionInstaller: The environment variable LD_LIBRARY_PATH is not set. It must contain the current folder '.'.
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
SNAP Release version 6.0-PREVIEW4
SNAP home: /opt/snap/bin//..
SNAP debug: null
SNAP log level: null
Java home: /opt/snap/jre
Java version: 1.8.0_102
Processors: 4
Max memory: 8.9 GB
Cache size: 1024.0 MB
Tile parallelism: 4
Tile size: 512 x 512 pixels
To configure your gpt memory usage:
Edit snap/bin/gpt.vmoptions
To configure your gpt cache size and parallelism:
Edit .snap/etc/snap.properties or gpt -c ${cachesize-in-GB}G -q ${parallelism}
tteyar@espvm30301:~/red/prog/fire/dun$ java -version
java version "1.8.0_144"
Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)
tteyar@espvm30301:~/red/prog/fire/dun$
The SNAP/gpt version seems to be the right one, and it seems to stay within its sand box using its own version of the java virtual machine (1.8.0_102), not the system-wide default (1.8.0.144).
Finally got to testing your latest (4.9.2017) graph. Unlike your screendump above, my result was the same error message as before ( …Cannot write multisize products.). Both on Windows (SNAP 5.0.8) and on Linux (SNAP 6.0.0).
Ciao
Yrjö
PS. In one of your postings above, you identify Big Geotiff as a source of slowing down processing. To clarify this, BIGTIFF as such is not a proble, the way SNAP writes BIGTIFF is. I compared gadlwarp with and without the command argument ‘-co “BIGTIFF=YES”’, and the excecution times were practically identical for a 9000x9000 image (actually BIGTIFF faster: 20.563 s, ClassicFIFF: 21.692 s).
PS. Marcus, if the recommendation is always to use BEAM-DIMAP and to add another gpt run for conversion, is this recommendation reflected in all tutorials and documentation of the graph engine of SNAP ?
With saying that the Bigtiff is the cause for the slowness I just wanted to say that it is not caused by the resampling. And the slowness is caused by the the implementation within SNAP for sure.
The recommendation to use BEAM-DIMAP applies only for intermediate results and not for the final output. The final result can be written to any format you want. But for intermediate results BEAM-DIMAP is recommended in order to retain all necessary information.
