Pixel Information in Exported TIFF File

MCiutys · January 23, 2018, 10:16pm

Hello everyone,

I am a newbie in remote sensing and Sentinel, so I apologize if my question looks very basic to you, but I could not find information anywhere.

My final year project involves some remote sensing and for some processing, I chose SNAP Toolbox. I downloaded a number of pictures from Scihub, resampled them and chose a subset of bands that relevant to me. I decided to export them as TIFF files. By doing this, I assumed that this file contains all the information (longitude, latitude, band data, etc.) for each pixel. So, my question is is that assumption correct?

As part of the project, I have to process TIFF files and I cannot extract band data which is essential to me. I just want to know which part is incorrect (either TIFF does not contain the information for each pixel or my program is incorrect). Thanks in advance.

Kranthi145 · January 24, 2018, 4:35am

Hi I am Also Doing My Final Year Project In Earth Observation.As Of I Now The Measurement Data Set In Meta Data Will Provide Time,Quality Flag,Image Data [Pixel Information Along With Bands And Intensity , Amplitudes]. I Will Send My Elsevier Paper To You.So That You Will Get An Idea About You.Please Send Me Your Base Paper So That It Will Help Me.Paper.pdf (2.2 MB)

gnwiii · January 24, 2018, 1:50pm

The TIFF file format is very flexible. In particular, you can create TIFF files that have colour “channels”, for example, 8-bit arrays for red, blue, green, and alpha, or you can create TIFF files that have data arrays as integers of various sizes or as floating point numbers. In addition, TIFF allows “tags” that provide metadata. GeoTIFF files use tags to provide the details the map projection, SNAP may use additional tags to specify band names and missing value codes, but not all 3rd party applications support these tags (and some refuse to load files that have “unrecognized” tags.

The GDAL software provides a gdalinfo program that can be run in a terminal to see the metadata a TIFF file. Many GUI applications, including SNAP, also provide ways to display metadata in TIFF files.

TIFF file exported from the window in SNAP6:

$ gdalinfo A2006167181000_L2_OC_chlor_a.tif
Driver: GTiff/GeoTIFF
Files: A2006167181000_L2_OC_chlor_a.tif
Size is 1354, 2020 
Coordinate System is `'
Image Structure Metadata:
  INTERLEAVE=PIXEL
Corner Coordinates:
Upper Left  (    0.0,    0.0)
Lower Left  (    0.0, 2020.0)
Upper Right ( 1354.0,    0.0)
Lower Right ( 1354.0, 2020.0)
Center      (  677.0, 1010.0)
Band 1 Block=1354x8 Type=Byte, ColorInterp=Red
  Mask Flags: PER_DATASET ALPHA 
Band 2 Block=1354x8 Type=Byte, ColorInterp=Green
  Mask Flags: PER_DATASET ALPHA 
Band 3 Block=1354x8 Type=Byte, ColorInterp=Blue
  Mask Flags: PER_DATASET ALPHA 
Band 4 Block=1354x8 Type=Byte, ColorInterp=Alpha

Exporting as geoTIFF:

$ gdalinfo A2006167181000_L2_OC_chlor_a_geo.tif
Driver: GTiff/GeoTIFF
Files: A2006167181000_L2_OC_chlor_a_geo.tif
Size is 1354, 2020
Coordinate System is:
GEOGCS["WGS 84",
    DATUM["WGS_1984",
        SPHEROID["WGS 84",6378137,298.257223563,
            AUTHORITY["EPSG","7030"]],
        AUTHORITY["EPSG","6326"]],
    PRIMEM["Greenwich",0],
    UNIT["degree",0.0174532925199433],
    AUTHORITY["EPSG","4326"]]
Origin = (-94.631786346435547,45.830932617187500)
Pixel Size = (0.067039489746094,-0.000389099121094)
Metadata:
  AREA_OR_POINT=Area
  TIFFTAG_RESOLUTIONUNIT=1 (unitless)
  TIFFTAG_XRESOLUTION=1
  TIFFTAG_YRESOLUTION=1
Image Structure Metadata:
  INTERLEAVE=PIXEL
Corner Coordinates:
Upper Left  ( -94.6317863,  45.8309326) ( 94d37'54.43"W, 45d49'51.36"N)
Lower Left  ( -94.6317863,  45.0449524) ( 94d37'54.43"W, 45d 2'41.83"N)
Upper Right (  -3.8603172,  45.8309326) (  3d51'37.14"W, 45d49'51.36"N)
Lower Right (  -3.8603172,  45.0449524) (  3d51'37.14"W, 45d 2'41.83"N)
Center      ( -49.2460518,  45.4379425) ( 49d14'45.79"W, 45d26'16.59"N)
Band 1 Block=1354x1 Type=Byte, ColorInterp=Red
  Mask Flags: PER_DATASET ALPHA 
Band 2 Block=1354x1 Type=Byte, ColorInterp=Green
  Mask Flags: PER_DATASET ALPHA 
Band 3 Block=1354x1 Type=Byte, ColorInterp=Blue
  Mask Flags: PER_DATASET ALPHA 
Band 4 Block=1354x1 Type=Byte, ColorInterp=Alpha

You can see that this added geolocation, but you still have an RGBA image.

If you load one of these files in SNAP6 the metadata display will present much the same information in a different format. If your files are like this, you will need to adjust your workflow. If you provide more details of what you have done, someone may be able to suggest changes. There are also many tutorials
on the ESA STEP site that may give you some ideas of how to proceed.

MCiutys · January 24, 2018, 10:50pm

Thank you for your answer, but I still have a few questions.

I was able to have a look at metadata using SNAP. The information is provided in a similar way as in your examples. However, what I noticed is that the information in metadata is relevant for the whole image and not distinct pixels (for example, image width or image length). I believe that “tags” provide information about the image, not pixels.

For my project, I need information (band value, to be exact) for every pixel as I have to loop through them and do some checks on each of them. The question is does TIFF file hold that information for each pixel? Or does it have only a general information about image that can be accessed using “tags”? If it does contain information for each pixel, is there any way to extract it? If it does not, maybe there is another way to output the image so I could access individual pixels? The file format does not have to be strictly TIFF, it can be anything, but I just need to have an access to each pixel.

I apologize if I understood your comment completely wrong.

gnwiii · January 25, 2018, 12:43pm

The first step is to determine what sort of tiff file you have. If you have three or four bands of type=“byte” then you probably have an RGB or RGBA image, e.g., colours, not data. Data usually will be in bands of type=“float32”, but can be stored as integers (in which case you may need to apply a scale transformation). Here is the gdalinfo output for a level-2 swath file:

 Driver: GTiff/GeoTIFF
 Files: /Volumes/data1/ocssw/benchmark/subset_A2006167181000.L2_OC.tif
 Size is 1354, 2020
 Coordinate System is:
 GEOGCS["WGS 84",
     DATUM["WGS_1984",
         SPHEROID["WGS 84",6378137,298.257223563,
             AUTHORITY["EPSG","7030"]],
         AUTHORITY["EPSG","6326"]],
     PRIMEM["Greenwich",0],
     UNIT["degree",0.0174532925199433],
     AUTHORITY["EPSG","4326"]]
 Origin = (-94.631786346435547,45.830932617187500)
 Pixel Size = (0.067039489746094,-0.000389099121094)
 Metadata:
   AREA_OR_POINT=Area
   TIFFTAG_IMAGEDESCRIPTION=subset_A2006167181000.L2_OC
   TIFFTAG_RESOLUTIONUNIT=1 (unitless)
   TIFFTAG_XRESOLUTION=1
   TIFFTAG_YRESOLUTION=1
 Image Structure Metadata:
   INTERLEAVE=BAND
 Corner Coordinates:
 Upper Left  ( -94.6317863,  45.8309326) ( 94d37'54.43"W, 45d49'51.36"N)
 Lower Left  ( -94.6317863,  45.0449524) ( 94d37'54.43"W, 45d 2'41.83"N)
 Upper Right (  -3.8603172,  45.8309326) (  3d51'37.14"W, 45d49'51.36"N)
 Lower Right (  -3.8603172,  45.0449524) (  3d51'37.14"W, 45d 2'41.83"N)
 Center      ( -49.2460518,  45.4379425) ( 49d14'45.79"W, 45d26'16.59"N)
 Band 1 Block=1354x2020 Type=Float32, ColorInterp=Gray
 Band 2 Block=1354x2020 Type=Float32, ColorInterp=Undefined
 Band 3 Block=1354x2020 Type=Float32, ColorInterp=Undefined
 Band 4 Block=1354x2020 Type=Float32, ColorInterp=Undefined

Normally, data for each band is stored in an array or 2-D table For “swath” data the geolocation may be given as locations for each pixel in arrays of longitudes and latitudes, or more compactly as location of some pixels with interpolation to compute locations of the other pixels. There is metadata that describes the whole file and also metadata for each band (called “properties” in SNAP).

You can save band data to GeoTIFF in a way that preserves data values, but metadata may be lost (because it is stored in TIFF tags that many applications ignore). If you use Export from the SNAP File menu, and choose GeoTIFF, you should get an image with Type=Float32, ColorInterp=Gray but you may not get band names, missing value codes, etc. in 3rd party applications. For “swath” data you should get bands for longitude and latitude of each pixel, but software intended for use with mapped data may not work properly or may refuse to load the files.

“Mapped” data have been projected onto a flat surface (map). Cylindrical equidistant projections are vry popular because you can easily compute the locations of pixels in software that doesn’t support map projections.

You combine (“Mosiac”) mapped data from multiple satellite passes, but some onput pixels may be omitted in areas where one mapped pixel corresponds several swath pixels, or replicated in areas where one swatch pixel corresponds several mapped pixels. Binning was devised as a way to ensure that each input pixel contributes equally to the mapped data.

In SNAP, you can do calculations that use values from one or more bands to create new bands ( “Band Maths”). The BEAM dimap format is often used to save data after doing some processing so that you can load it and continue with more processing rather than always restarting from the original file. The dimap format stores metadata in the .dim file and an image for each band in the .data folder.

The NetCDF-CF format has a robust metadata standard and is well-suited to mapped data (regular grids), but older software may not support recent NetCDF versions.

MCiutys · January 25, 2018, 4:53pm

Thank you for your answer. So, I exported the file as GeoTIFF and there is the gdalinfo output I got:

    Driver: GTiff/GeoTIFF
    Files: check.tif
    Size is 1830, 1830
    Coordinate System is:
    PROJCS["WGS 84 / UTM zone 29N",
        GEOGCS["WGS 84",
            DATUM["WGS_1984",
                SPHEROID["WGS 84",6378137,298.257223563,
                    AUTHORITY["EPSG","7030"]],
                AUTHORITY["EPSG","6326"]],
            PRIMEM["Greenwich",0],
            UNIT["degree",0.0174532925199433],
            AUTHORITY["EPSG","4326"]],
        PROJECTION["Transverse_Mercator"],
        PARAMETER["latitude_of_origin",0],
        PARAMETER["central_meridian",-9],
        PARAMETER["scale_factor",0.9996],
        PARAMETER["false_easting",500000],
        PARAMETER["false_northing",0],
        UNIT["metre",1,
            AUTHORITY["EPSG","9001"]],
        AUTHORITY["EPSG","32629"]]
    Origin = (600000.000000000000000,5900040.000000000000000)
    Pixel Size = (60.000000000000000,-60.000000000000000)
    Metadata:
      AREA_OR_POINT=Area
      TIFFTAG_IMAGEDESCRIPTION=check
      TIFFTAG_RESOLUTIONUNIT=1 (unitless)
      TIFFTAG_XRESOLUTION=1
      TIFFTAG_YRESOLUTION=1
    Image Structure Metadata:
      INTERLEAVE=BAND
    Corner Coordinates:
    Upper Left  (  600000.000, 5900040.000) (  7d30' 5.54"W, 53d14'24.75"N)
    Lower Left  (  600000.000, 5790240.000) (  7d32' 6.07"W, 52d15'12.44"N)
    Upper Right (  709800.000, 5900040.000) (  5d51'29.63"W, 53d12'29.52"N)
    Lower Right (  709800.000, 5790240.000) (  5d55'41.96"W, 52d13'21.24"N)
    Center      (  654900.000, 5845140.000) (  6d42'20.80"W, 52d44' 2.09"N)
    Band 1 Block=1830x1830 Type=Float32, ColorInterp=Gray
    Band 2 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 3 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 4 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 5 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 6 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 7 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 8 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 9 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 10 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 11 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 12 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 13 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 14 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 15 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 16 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 17 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 18 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 19 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 20 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 21 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 22 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 23 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 24 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 25 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 26 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 27 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 28 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 29 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 30 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 31 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 32 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 33 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 34 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 35 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 36 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 37 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 38 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 39 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 40 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 41 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 42 Block=1830x1830 Type=Float32, ColorInterp=Undefined
    Band 43 Block=1830x1830 Type=Float32, ColorInterp=Undefined

So this file does seem to have band data. For dealing with the TIFF files, I am using C++ and LibTIFF library. At the moment, my code looks like this:

TIFF* tif = TIFFOpen("check.tif", "r");
	    if (tif) {
		uint32 imagelength;
		uint32 i;
		float* data;
		tdata_t buf;
		uint32 row;
		TIFFGetField(tif, TIFFTAG_IMAGELENGTH, &imagelength);
		buf = _TIFFmalloc(TIFFScanlineSize(tif));

		for (row = 0; row < imagelength; row++) {
		    TIFFReadScanline(tif, buf, row);

		    data = (float*) buf;

		    for (i = 0; i < 1830; i++) {
		    	for (int j = 0; j < 2; j++) {
			    	cout << "Row: " << row << ", column: " << i << ": " << data[i] << endl;
		    	}
		    }

		    if (row == 0) break;

		}
		_TIFFfree(buf);
		TIFFClose(tif);
	    }

The pixel values I get when printing out are simple integers, such as 1467, 1471, 1514, etc. and those are clearly not the band values. So how are band values stored for each pixel? Is it in an array? I tried to put one more loop inside to loop through 43 elements, but my program crashed. Maybe LibTIFF is not suitable for this and there is other specific library?

I know this is not too much related to SNAP Toolbox, but I cannot find any information about it on the Internet. Thank you once again.

gnwiii · January 25, 2018, 5:47pm

Now you are suffering from the limited metdata support in geoTIFF. It is possible that the band in question is stored as integers and has some associated scaling to floating point. This is very common in remote sensing. The SNAP Help for “Import and Export/Export raster data/GeoTIFF” lists the metadata that should be stored in tags known to SNAP. Can you load the file in SNAP? Do you get meaningful band names and data values?

If so, the information must be in the file, but you will need add support for those tags to your library (or wirte code that will only work for your specific file). You may find it easier to work with netCDF files. There is a wealth of information at the Unidata NetCDF pages and also at the HDFGroup (NetCDF4 is a subset of HDF5, so HDF5 tools are often useful when working with NetCDF4 files).

aditi.iirs · September 6, 2023, 10:04am

I have a metadata file in json format. I tried to convert it into xml, but the SAR data is not getting called thorugh the xml file while trying to open it. Its an UMBRA dataset. Can anyone give an insight into it?

raavikant · September 7, 2023, 5:55am

Hi,

Your assumption is partially correct. TIFF files can indeed contain all the necessary information for each pixel, including band data. However, the way the data is stored in the TIFF file depends on how you exported it from SNAP Toolbox.

Here are some steps to help you diagnose the issue:

Check the Export Settings: Ensure that when you exported the images as TIFF files from SNAP Toolbox, you selected the option to include band data. In some cases, you might need to specify the data format and other settings during the export process.
Image Metadata: Open the TIFF files using an image viewer or software that can display metadata. Look for metadata information within the TIFF file, such as geospatial coordinates (latitude and longitude) and band information. If this information is present, your TIFF files contain the required data.
Image Viewer: Use a dedicated remote sensing image viewer or GIS software (e.g., QGIS, ArcGIS) to open the TIFF files. These tools should allow you to access and visualize the band data. If you can view the bands, then the data is present in the TIFF files.
Data Extraction: If you are still unable to extract band data, review the processing steps within SNAP Toolbox. Ensure that you correctly subset the bands you need and export them with the necessary settings.
Consult Documentation: Refer to the documentation of the Sentinel-1 data and SNAP Toolbox to understand the export options and data formats in more detail.

In summary, TIFF files can store the information you need, but the correct export settings and data organization are crucial. Verify your export settings and use appropriate software to access and extract the band data from your TIFF files.

aditi.iirs · September 7, 2023, 1:10pm

Hi Raavikant,
Thank you for the reply.
The TIFF file is a processed dataset, that does not have the real and imaginary complex information. I need the complex data, as I am working towards Interferometric application.
My previous query was not very detailed, that was my entirely my mistake.
I am working through the documentation.