Writing out a product as GeoTIFF-BigTIFF

development python
#1

Hi,

I’ve had success using the example files to open/read data from S2 tile products then manipulate the data in python but now I am trying to write the data (+ metadata/geocoding) to GeoTIFF-BigTIFF I am getting an unattribute java nullpointerexception.

In brief my band/pixel writing code looks like this:

write out the band rasters to new products

        blended_tile_1 = Product(tile1_name + '_blended', 'reflectance',
                                 tile_width, tile_height)
        blendedTile_band = blended_tile_1.addBand(band_names[8],
                                                  ProductData.TYPE_FLOAT32)
        prodWriter = ProductIO.getProductWriter('GeoTIFF-BigTIFF')
        ProductUtils.copyGeoCoding(p1, blended_tile_1)
        blended_tile_1.setProductWriter(prodWriter)
Snappy_ndvi export failure
#2

??? why did that post?

   >     r8a = np.zeros(tile_width, dtype=np.float32)

        for y in range(tile_height):
            r8a = rad8a_1.readPixels(0, y, tile_width, 1, r8a)
            blendedTile_band.writePixels(0, y, tile_width, 1, r8a)

So for test purposes I am just transferring data from my input to a new output file and band. From the NDVI example I noticed I am missing a writeHeader call but this seemed to be specific to DIMAP.
Any ideas what I shold do / the problem with writing to GeoTIFF?

#3

Try using:

ProductIO.writeProduct(<Your product>, <Output path>, "GeoTIFF-BigTIFF")
#4

No, the writeHeader is also needed for GeoTiff. If I remember correctly. At least before writing the band data.
If you just want to copy bands, it is best practice to use ProductUtils.copyBand(…)
And than use ProductIO.writeProduct(…), what @Ciaran_Evans suggested.
If you want to compute something you might consider to implement an operator.
https://senbox.atlassian.net/wiki/spaces/SNAP/pages/42041346/How+to+write+a+processor+in+Python

#5

Thanks for the advice and adding writeHeader did create a file stub but did not solve the problem. I am still getting a nullpointerexception when I attempt to wrote pixels to the band added to the product.

I actually need to modify the band data before writing out so copyBand and ProductIO.writeProduct(…) won’t work if I have not created my new band data in the product. I am finding the API confusing as there seem to be several different ways to write / set information. For instance in my example I thought I constructed a product with the name 'tile1_name + ‘_blended’ but the file created took the name from
blended_tile_1.writeHeader(‘snappy_test_output.tif’)

#6

Hi Marco,
OK the writeHeader is creating a file and I tried it also with BEAM-DIMAP and the header looked good. The behaviour is not linked to the GeoTIFF export as I get the same failure with DIMAP. I get an unattribute java nullpointerexception on the first attempt to ‘setPixels’

        > prodWriter = ProductIO.getProductWriter('BEAM-DIMAP')  #'GeoTIFF-BigTIFF')

        ProductUtils.copyGeoCoding(p1, blended_tile_1)
        blended_tile_1.setProductWriter(prodWriter)
        blended_tile_1.writeHeader('snappy_test_output.dim')    #tif')

        r8a = np.zeros(tile_width, dtype=np.float32)

        for y in range(tile_height):
            r8a = rad8a_1.readPixels(0, y, tile_width, 1, r8a)
            blendedTile_band.setPixels(0, y, tile_width, 1, r8a)

I have check all of the relevant variables (the ra8 array, the band object, tile_width) and all are valid, not null so i don’t know where the nullpointerexception is coming from. I’ve tried to follow the examples from other Forum messages about exporting using snappy and seem to be following the same API usage. I only copy an input band for test purposes, my goal is to write out altered pixel values.

#7

Hello,
I would like to sequentially write pixel values (product.writePixels(…)) in netCDF4 (or other format).
To do this I need to construct the image structure beforehand with product.writeHeader(‘test.nc’).
But, the resulted image test.nc is unreadable (by snap or gdal…).
Here is a small piece of code to check the consistency of the different format within snappy:

import numpy as np
from esasnappy import Product, ProductIO, ProductData

formats = ([".dim", "BEAM-DIMAP"],
           [".h5", "HDF5"],
           [".img", "GDAL-HFA-WRITER"],
           [".jp2", "JPEG2000"],
           [".bin", "Generic Binary BSQ"],
           [".tif", "GeoTIFF+XML"],
           [".grd", "GDAL-GSBG-WRITER"],
           [".nc", "NetCDF-CF"],
           [".nc", "NetCDF-BEAM"],
           [".nc", "NetCDF4-BEAM"],
           [".tif", "GeoTIFF"],
           ["", "JP2"],
           [".tif", "GDAL-GTiff-WRITER"],
           [".tif", "GeoTIFF-BigTIFF"],
           [".csv", "CSV"],
           [".bmp", "GDAL-BMP-WRITER"],
           [".rst", "GDAL-RST-WRITER"],
           [".nc", "NetCDF4-CF"],
           [".hdr", "ENVI"])

for format in formats:
    print()
    print('---------------')
    print(format)
    file = 'issues/test' + format[0]

    product = Product('test', 'test', 100, 100)
    band = product.addBand('SZA', ProductData.TYPE_FLOAT32)
    band.setModified(True)
    band.setNoDataValue(np.nan)
    band.setNoDataValueUsed(True)
    product.getBand('SZA').setDescription('Solar zenith angle in deg.')

    writer = ProductIO.getProductWriter(format[1])

    product.setProductWriter(writer)
    try:
        product.writeHeader(file)

        p = ProductIO.readProduct(file)
        try:

            print(p.getBand('SZA').getDescription())
            print('SUCCESS!!!!!!!')
        except:
            print('error for format ', format[1])
    except:
        print('no header allowed for', format[1])
    print()

with the following results:

---------------
['.dim', 'BEAM-DIMAP']
WARNING: org.esa.snap.core.dataio.dimap.DimapProductReader: DimapProductReader: Unable to read file 'issues/test.data/SZA.img' referenced by 'SZA'.
WARNING: org.esa.snap.core.dataio.dimap.DimapProductReader: DimapProductReader: Removed band 'SZA' from product 'null'.
Solar zenith angle in deg.
SUCCESS!!!!!!!


---------------
['.h5', 'HDF5']
no header allowed for HDF5


---------------
['.img', 'GDAL-HFA-WRITER']
no header allowed for GDAL-HFA-WRITER


---------------
['.jp2', 'JPEG2000']
no header allowed for JPEG2000


---------------
['.bin', 'Generic Binary BSQ']
first_line_time metadata value is null
last_line_time metadata value is null
error for format  Generic Binary BSQ


---------------
['.tif', 'GeoTIFF+XML']
first_line_time metadata value is null
last_line_time metadata value is null
Solar zenith angle in deg.
SUCCESS!!!!!!!


---------------
['.grd', 'GDAL-GSBG-WRITER']
no header allowed for GDAL-GSBG-WRITER


---------------
['.nc', 'NetCDF-CF']
error for format  NetCDF-CF


---------------
['.nc', 'NetCDF-BEAM']
error for format  NetCDF-BEAM


---------------
['.nc', 'NetCDF4-BEAM']
error for format  NetCDF4-BEAM


---------------
['.tif', 'GeoTIFF']
Solar zenith angle in deg.
SUCCESS!!!!!!!


---------------
['', 'JP2']
first_line_time metadata value is null
last_line_time metadata value is null
error for format  JP2


---------------
['.tif', 'GDAL-GTiff-WRITER']
no header allowed for GDAL-GTiff-WRITER


---------------
['.tif', 'GeoTIFF-BigTIFF']
no header allowed for GeoTIFF-BigTIFF


---------------
['.csv', 'CSV']
no header allowed for CSV


---------------
['.bmp', 'GDAL-BMP-WRITER']
no header allowed for GDAL-BMP-WRITER


---------------
['.rst', 'GDAL-RST-WRITER']
no header allowed for GDAL-RST-WRITER


---------------
['.nc', 'NetCDF4-CF']
error for format  NetCDF4-CF


---------------
['.hdr', 'ENVI']
no header allowed for ENVI

Conclusion product.writeHeader() is efficient for GeoTIFF and Beam-dimap only.
Any misunderstanding from my part? or ideas for workaround?
Thanks
TRistan

1 Like
#8

I have the same error, how did you solve it?

Thanks