With small areas or 20-m pixel spacing, this works. When we try to apply it over a 100-km tile of Sentinel-2 data (bands B2,B3,B4,B8) with pixel spacing of 10 m in Linux computers, it does not complete. On my desktop Linux computer (16 GB of memory), the CPU load approaches 100 percent, and the fans inside the computer start making clearly elevated noise. Memory allocation of process “java” is slightly over 10 GB, which is somewhat more than the expected size of the output image (64-bit, 8-band Geotiff-BIGTIFF file - the process writes the beginning part of about 20 MB, so I know these file details). The operating system does not start swapping activity. Input is a 4-band Geotiff-BIGTIFF file. On the F-TEP servers, I cannot hear the noise, but the job does not complete within half a day.
Trying the equivalent mosaicking operation over the same image data but using the GUI on my Windows computer produces the output image in about an hour. gpt refuses to co-operate in this computer, so I cannot test if it is an operating-system related problem.
Does anybody know, what could be the problem ? SNAP version is 5.0.8 in my Linux computer and 5.0.0 in the F-TEP servers.
I tested with SNAP 6.0.0 also: the same result, no output in any decent time (just the tiff header in about an hour) using gpt. I also tested with the SNAP GUI: Raster/Geometric Operations/Mosaicing. The run completed in about an hour producing an 8-band Geotiff-BIGTIFF file (64 bits per pixel/band) of 5 878 602 329 bytes (almost 6 GB).
Output of tiffdump (after SNAP GUI, the gpt version had TileOffsets and TileByteCounts zeroed):
Could it be that - despite writing a valid BIGTIFF header - the Mosaic module used by gpt is not handling 64-bit variables properly ? Should the GUI/Mosaicing use the same operator as gpt when given the graph at the beginning of this thread ?
The theory about possible 64-bit problems seemed to be wrong. I made today a test by shrinking the area of interest so that the output file is about 1) 2.05 GB, and 2) about 3.8 GB. Test 1) went smoothly, but test 2) did not complete. So, it seems that SNAP/gpt/Mosaic has some internal bottle-neck that prevents its use on large images. Maybe we must make the re-projection/mosaicking operations outside SNAP/gpt e.g. by using gdalwarp and gdal_merge.py (which are not perfect either when fed with large data volumes, but they should work with this size of dataset).
It might be that the cause of this issue is the GeoTiff format. We have also seen some problems with this.
Would it be possible for you to try the same with the BEAM-DIMAP format and only change the format to GeoTIFF as last step.
I know that we have created bigger mosaic images as 10000x10000. So I think the problem is not the Mosaic operation it’s self.
That processing behaves differently in the GUI and on the command line has been see before (SNAP-640). Hopefully we can address it in the next dev cycle.