Hi, I’m using GPT to reproject a product to create a georectified dataset.
I’m using GeoTIFF as output format. Since the output could be big (as an instance 20K x 10K pixels) I would like to write it tiled so I can achieve better performance when accessing it back.
(See TIFF Specification 6.0, Section 15, page 66)
Does the writeOp in the GPF graph support any kind of parameter to achieve that?
Does the SNAP GeoTIFF writer support inner tiling (as well as other params like compression)?
The data is processed and written tile-wise by default. The tile-size of the product is used when it is written.
Be default the tile size is 512x512. But this can overwrite each product reader separately.
You can specify the system properties snap.dataio.reader.tileWidth and snap.dataio.reader.tileHeight when you invoke the gpt.
e.g. -Dsnap.dataio.reader.tileWidth=* -Dsnap.dataio.reader.tileHeight=10 to write always chunks of 10 complete lines.
The asterisk means the full width or full height respectively .
Explicit write parameters are not yet supported but we have it on the agenda.
Hi,
thanks for the feedbacks.
However, what if I’m starting from a NetCDF product?
I’m setting up a graph which does this steps
READ (a not georectified NetCDF)
SUBSET (select a reduced number of bands)
REPROJECT (fixed resolution and WGS84 CRS)
WRITE (to GeoTIFF format)
NetCDF products are untiled. I have tried those system properties right now but the output is still untiled.
Is it there any other param or operation to force the tiling across the graph?
Sorry, I was wrong. The GeoTiff writer does not write in tiles.
But you Can use the BigTIFF.
Specify as target format GeoTIFF-BigTIFF
As system parameter when invoking gpt use the following parameters
-Dsnap.dataio.bigtiff.tiling.width=<number>
-Dsnap.dataio.bigtiff.tiling.height=<number>
Hi Marco,
Thanks for the suggestion. I’ll use that driver in the meantime.
Any plan to “unify” the dependencies and use the same low level writer for both geotiff and bigtiff?
At the end, I think the only difference will be setting the forceBigTiff flag to false or true.
For the BigTiff it is possible.
Set the Java system property -Dsnap.dataio.bigtiff.compression.type=LZW
LZW is the only supported compression type. Be default it is not set and no compression is applied.
This was by intention. At the time of writing the BigTiff writer several tools had problems opening the written files.
Additionally you can set the compression quality: -Dsnap.dataio.bigtiff.compression.quality=0.9
The default value is 0.75. Quality value must be between 0.0 and 1.0
This a little of topic but still closely relates to the tiling problem.
When writing GeoTiff files, by default the tile size is number of samples x number of lines. I.e., the whole image is written as one whole block.
This is neither tile-based nor line-based and thus extremely slow to read by other software (I tested ArcMap and Envi).
In a classical line-based approach the block size is number of samples x 1.
Converting the image via e.g. gdal_translate speeds up reading operations significantly.
this is how it looks like in gdalinfo:
tile-based: Band 1 Block=256x256 Type=Float32, ColorInterp=Gray
line-based: Band 1 Block=16635x1 Type=Float32, ColorInterp=Gray
snap GeoTiff: Band 1 Block=16635x10662 Type=Float32, ColorInterp=Gray
Is there a specific reason for doing it like this?
By default the product which shall be written is asked for it’s tiling and this is used for the tiling of the GeoTiff file. If the product has this kind of tiling, as you have described, then it is written like this. So by default the tiling is preserved. And if is source product comes without a default tiling, then we apply a tiling of 512x512.
The question is how the data you are writing got this kind of tiling.
What are your input files and which processing steps have you performed? Maybe something got wrong in between.
Thanks a lot for your quick reply. I am processing Sentinel-1 GRD images. The tiling of the original tiff files is line-based. Preserving the tiling would be good in this case but does not happen for some reason.
Related: the geotiff driver exports files with INTERLEAVE=PIXEL . Because S3 has many bands, it takes longer read bands at the end. It would be better if the creation parameter was INTERLEAVE=BAND (see http://www.gdal.org/frmt_gtiff.html)
Can this be supported as another tiff creation option?
We miss this feature to parameterise the writer and reader. It is actually a very useful feature. WE have it on the development plan, but I can’t give you a time schedule for it.
Hello Marc,
I see that among the output formats now supported there is also GDAL-GTiff-WRITER.
Does it also support tiling, compression etc.? GDAL has options to do so, can such options be passed?
I ask so because
using GeoTIFF-BigTIFF takes a lot of more time for writing with respect to GeoTIFF+XML
if I save manually the GeoTIFF, it is usually smaller than 4GB
Geotiff with geocoding information is just Tiff. Your welcome!
Maybe you tell a bit more.
Is this the problem? That you don’t have geo-information attached to the GeoTiff file after you have converted something?
Could you tell what your input data is and how you create the GeoTiff file? Are you performing some processing step?