Error on writing target product

Hello all.

I’m quietly newbie in snappy, and I’ve stucked in writing new product when I trained myself.

The issue is that an error “RuntimeError: java.lang.IllegalStateException: no product reader for band ‘Enhanced_VV’” when try to write product using ProductIO.writeProduct() method.

I’ve tried to do some tutorials and not yet experienced this issue, hence I think there are some flaws in my source code.

My source code do pretty simple work, makes a new band ‘Enhanced_VV’ to target product, and simply copy ‘Intensity_VV’ band of source product to ‘Enhanced_VV’

The source code as follows:

def compute(self):
width = self.source.getSceneRasterWidth()
height = self.source.getSceneRasterHeight()

name = self.source.getName()
description = self.source.getDescription()
band_names = self.source.getBandNames()

print("Product: %s, %d x %d pixels, %s" % (name, width, height, description))
print("Bands:   %s" % (list(band_names)))

b = self.source.getBand('Intensity_VV')
target_product = snappy.Product(self.source.getProductType(), self.source.getProductType(), width, height)
# a
snappy.ProductUtils.copyProductNodes(self.source, target_product)

target_band = target_product.addBand('Enhanced_VV', snappy.ProductData.TYPE_INT32)
target_band.setNoDataValue(numpy.nan)
target_band.setNoDataValueUsed(True)

print self.target

writer = snappy.ProductIO.getProductWriter('BEAM-DIMAP')

r = numpy.zeros(width, numpy.int32)

snappy.ProductUtils.copyGeoCoding(self.source, target_product)

target_product.setProductWriter(writer)
target_product.writeHeader(snappy.String('output.dim'))

for y in range(height):
    b.readPixels(0, y, width, 1, r)
    target_band.writePixels(0, y, width, 1, r)

snappy.ProductIO.writeProduct(target_product, self.target, 'BEAM-DIMAP')

target_product.closeIO()
print 'Done.'

The probable line is ‘snappy.ProductIO.writeProduct(target_product, self.target, ‘BEAM-DIMAP’)’ of last thrid line.

An error gives me information about product reader for band, but I cannot guess what is wrong.

is there any suggestion to solving this issue?

Thank you in advance.

You got pretty far by training yourself. Very good.

The error which occurs is probably also caused by the misleading API. You have to read the doc very careful to get the difference.
The method writePixels directly writes the values to disk and by passes the band/product.
When you call ProductIO.writeProduct afterwards there is no source for the data which shall be written. Neither as buffer within the band nor a reader where the data could be read from.
So I would suggest to replace writePixels by setPixels.

...
snappy.ProductUtils.copyGeoCoding(self.source, target_product)

for y in range(height):
   b.readPixels(0, y, width, 1, r)
   target_band.setPixels(0, y, width, 1, r)

snappy.ProductIO.writeProduct(target_product, self.target, 'BEAM-DIMAP')

target_product.closeIO()
print ‘Done.’

2 Likes

It works, marpet, many thanks!
And I also appreciate your advice: read doc carefully (This advice helps me to solve another issue.)

The modified code as follows:

class EightBitStretching:

def __init__(self, source_product, target_directory):
    self.source = source_product
    self.target_dir = target_directory

    self.width = self.source.getSceneRasterWidth()
    self.height = self.source.getSceneRasterHeight()

    self.name = self.source.getName()
    self.type = snappy.ProductData.TYPE_INT32
    self.description = self.source.getDescription()
    self.band_names = self.source.getBandNames()

    print("Source Product: %s, %d x %d pixels, %s" % (self.name, self.width, self.height, self.description))
    print("Source Bands:   %s" % (list(self.band_names)))

    self.target = snappy.Product(self.source.getProductType(), self.source.getProductType(), self.width, self.height)
    snappy.ProductUtils.copyProductNodes(self.source, self.target)

def compute(self):
    b = self.source.getBand('Intensity_VV')

    target_band = snappy.Band('_Enhanced', self.type, self.width, self.height)
    snappy.ProductUtils.copyRasterDataNodeProperties(b, target_band)
    self.target.addBand(target_band)

    # target_band = self.target.addBand('Enhanced_VV', snappy.ProductData.TYPE_INT32)

    target_band.setNoDataValue(numpy.nan)
    target_band.setNoDataValueUsed(True)

    r = numpy.zeros(self.width, numpy.int32)

    self.target.setProductWriter(snappy.ProductIO.getProductWriter('BEAM-DIMAP'))
    self.target.writeHeader(snappy.String(self.name+'_Enhanced.dim'))

    print("Target Product: %s, %d x %d pixels, %s" % (self.target.getName(), self.target.getSceneRasterWidth(), self.target.getSceneRasterHeight(), self.target.getDescription()))
    print self.target.containsBand('_Enhanced')
    target_band.ensureRasterData()

    # snappy.ProductUtils.copyGeoCoding(self.source, self.target)
    for y in range(self.height):
        b.readPixels(0, y, self.width, 1, r)
        target_band.setPixels(0, y, self.width, 1, r)

    snappy.ProductIO.writeProduct(self.target, self.target_dir+'.dim', 'BEAM-DIMAP')
    self.target.closeIO()
    print 'Done.'

Since Band.setPixels() method throws an exception if it has no raster data, I added ‘target_band.ensureRasterData()’ to make sure existance of raster data. I think this method makes a new raster data if not exist.

Anyway, thanks to marpet, I can process next step :slight_smile:

2 Likes

Hi,

I am trying to write Sentinel-2 masked bands target products to one single GeoTIFF, but by following the same procedure and replacing the product writer to “GeoTIFF-BigTIFF” instead of “BEAM-DIMAP” and the header extension to .tif instead of .dim, I get the following error:

Traceback (most recent call last):
File “/home/s_admin/snappy_bands_with_mask.py”, line 293, in
Blist[i].writePixels(0, y, width, 1, ma.filled(numpy.nan))
RuntimeError: java.lang.IllegalStateException: no product reader for band ‘b2’

When I replace ‘writePixels’ by ‘setPixels’ and setting ‘band.ensureRasterData()’, the error is the same, except it does not give any reason, just the line:

RuntimeError: java.lang.IllegalStateException

Thank you for your help.

Hi,

Unfortunately this is a very misleading error message.

Most likely, the actual problem is that the band’s of S-2 products have different image sizes. The only format that supports writing such product with different size bands is BEAM-DIMAP. We are planning to add the multi-size support for other suitable product formats in the future (e.g. netCDF, HDF, multi-image GeoTIFF).

You could try writing out the individual bands as GeoTIFF, e.g. by using the GDAL Python binding or similar.

Regards
Norman

Thanks forman. I’ll stick to the beam-dimap format for the time being.

Regards!

Hello there,
I have to create a Stack of some S-1 products and, since I cannot solve the sorting issue related to the “CreateStackOp” presented in this thread (Call ProductSet-Reader and Create Stack in Snap) I’m trying to get the stack by myself.
I wrote a code but when I try to write the final product to file I get the error 'java.lang.IllegalStateException: no product reader for band ‘Sigma0_VV_20170103’.

This is my code:

...
files_list=glob.glob(InputFold+'\*.dim')

#Initialize snappy
HashMap = jpy.get_type('java.util.HashMap')
#Get snappy operators    
GPF.getDefaultInstance().getOperatorSpiRegistry().loadOperatorSpis()

ind= [m.start() for m in re.finditer('_',os.path.split(files_list[0])[1])]
date_start = os.path.split(files_list[0])[1][ind[3]+1:ind[3]+9]
ind= [m.start() for m in re.finditer('_',os.path.split(files_list[-1])[1])]
date_end = os.path.split(files_list[-1])[1][ind[3]+1:ind[3]+9]
#==============================================================================
# CREATE 2 PRODUCTS, ONE FOR SIGMA_VV AND ONE FOR SIGMA_VH
#==============================================================================
source = ProductIO.readProduct(os.path.join(InputFold,files_list[0]))
width=source.getSceneRasterWidth()
height=source.getSceneRasterHeight()
writer = ProductIO.getProductWriter('BEAM-DIMAP')

product1 = Product('Sigma0_VV_Stack_'+date_start+'_'+date_end, source.getProductType(), width, height)
product2 = Product('Sigma0_VH_Stack_'+date_start+'_'+date_end, source.getProductType(), width, height)

snappy.ProductUtils.copyGeoCoding(source, product1)
snappy.ProductUtils.copyGeoCoding(source, product1)
product1.setProductWriter(writer)
product2.setProductWriter(writer)
#==============================================================================
# LOOP OVER THE FILES
#==============================================================================
for file in files_list:
        ind= [m.start() for m in re.finditer('_',os.path.split(file)[1])]
        date = os.path.split(file)[1][ind[3]+1:ind[3]+9]
        # OPEN FILES
        source = ProductIO.readProduct(file)
        # SELECT BANDS WE WANT TO STACK
        band1 = source.getBand('Sigma0_VV')
        band2 = source.getBand('Sigma0_VH')
        # RENAME BANDS ADDING THE DATE 
        band1.setName(band1.getName()+'_'+date)
        band2.setName(band2.getName()+'_'+date)
        # ADD THE SELECTED BANDS TO THE PRODUCTS CREATED
        product1.addBand(band1)
        product2.addBand(band2)
        
#==============================================================================
# ADD 2 BANDS TO THE PRODUCTS, ONE WITH THE LAT, THE OTHER WITH THE LON OF EACH PIXEL 
#==============================================================================
BandDescriptor = jpy.get_type('org.esa.snap.core.gpf.common.BandMathsOp$BandDescriptor')

#Latitude Band
lat = BandDescriptor()
lat.name = 'Latitude'
lat.type = 'float64'
lat.expression = 'LAT'

#Longitude Band
lon = BandDescriptor()
lon.name = 'Longitude'
lon.type = 'float64'
lon.expression = 'LON'

newBands = jpy.array('org.esa.snap.core.gpf.common.BandMathsOp$BandDescriptor', 2)
newBands[0] = lat
newBands[1] = lon
#Compute BANDMATHS Operator        
parameters = HashMap()
parameters.put('targetBands', newBands)
prod1 = GPF.createProduct('BandMaths', parameters, source)
#Get new bands and add to the Products created
ProductUtils.copyBand('Latitude',prod1,product1,True)
ProductUtils.copyBand('Longitude',prod1,product1,True)
ProductUtils.copyBand('Latitude',prod1,product2,True)
ProductUtils.copyBand('Longitude',prod1,product2,True)
#==============================================================================
# WRITE PRODUCTS TO FILE 
#==============================================================================
ProductIO.writeProduct(product1,os.path.join(OutputFold,os.path.split(files_list[-1])[1][1:ind[3]]+'_Sigma0_VV_Stack_'+date_start+'_'+date_end), 'BEAM-DIMAP')
ProductIO.writeProduct(product2,os.path.join(OutputFold,os.path.split(files_list[-1])[1][1:ind[3]]+'_Sigma0_VH_Stack_'+date_start+'_'+date_end), 'BEAM-DIMAP')

Anyone has any suggestion to solve this issue?
Thank you in advance!

Davide

I guess it is caused by how you copy the Sigma0_VV band to the new product.
Try using the ProductUtils.copyBand method you already use further down.
There is a version of this method with which you can rename the band in one call.

public static Band copyBand(String sourceBandName, Product sourceProduct,
                            String targetBandName, Product targetProduct, boolean copySourceImage)

Thank you very much @marpet I followed your advice and now it works! :slight_smile:

Hi guys,
I have a little issue about the processing times of product writing.
This is the final part of my stacking code, where I write the final stacked products to file:

...
for prod in products:
                print('Writing product "{}"...'.format(prod.getName()))
                sys.stdout.flush()
                time.sleep(.1)
                ProductIO.writeProduct(prod,os.path.join(OutputFold,os.path.split(files_list[-1])[1][1:ind[3]]+'_'+prod.getName()), 'BEAM-DIMAP')
                print('Product "{}" wrote! - t = {:.2f}s'.format(prod.getName(),time.time()-st))

Let’s take for example the same set of products to stack.
Now if I stack only 1 band (e.g.: ‘B2’ or ‘B3’) for this set it takes about 10 s to write.
If I add another band -> 2 total bands to stack (e.g.: [‘B2’ , ‘B3’]) it takes about 10 s for the first band and about 100 s for completing the process…why does it take so long to write the second product compared to the fisrt?
If I add other bands the result is the same: about 10 s for each band except the last one, and around 100 s for the last one :confused:
This is what I got printed with 4 bands:

Writing product "B2_Stack_20170102_20170621"...
Product "2A_MSIL1C_20170621T110651_N0205_B2_Stack_20170102_20170621" wrote! - t = 8.93s
Writing product "B3_Stack_20170102_20170621"...
Product "2A_MSIL1C_20170621T110651_N0205_B3_Stack_20170102_20170621" wrote! - t = 18.34s
Writing product "B4_Stack_20170102_20170621"...
Product "2A_MSIL1C_20170621T110651_N0205_B4_Stack_20170102_20170621" wrote! - t = 27.86s
Writing product "B8_Stack_20170102_20170621"...
Product "2A_MSIL1C_20170621T110651_N0205_B8_Stack_20170102_20170621" wrote! - t = 118.81s
Done! - t = 118.81s

Can anyone help me out with this issue?
Thank you very much!

That’s strange. Which processing steps are you before writing the data?

Well the behaviour described above seems not to be always respected somehow:
sometimes it completes the stacking in a total time equal to the sum of each single band stacking time;
some other times it takes so much more time (usually for the last band);
sometimes it even takes a lot of time for doing only a stack of 1 single band…
These cases always using the same files.
I don’t know why this behaviour, I don’t think that my CPU or RAM usage was affecting the results so much but I can’t be sure about it.

Anyway this is the code I did for stacking (for anyone who might want to use it as well)

def stack(InputFold=None, OutputFold = None, bnames = None):
        '''stack(InputFold = None,OutputFold = None, bnames = None) makes you select an input directory InputFold and creates a time stack with all the ".dim" files in that directory.
        One stacked product is created for each band in bnames (The band is required to be presented in all the files in the InputFold).
        The new products are stored in the output directory OutputFold.
        If no OutputFold is given, the function will create it in the same parent directory of the InputFold directory, and will name it by appending '_STACK' to the InputFold name.
        In addition to the bands at different times, for each product 2 bands with LAT and LON values per pixel are created.'''
        import os
        import snappy
        import re
        from snappy import HashMap, ProductIO, GPF, jpy, Product, ProductUtils
        import glob
        import tkinter as tk
        from tkinter import filedialog
        import time
        import sys
                
        if InputFold is None:
                root=tk.Tk()
                root.call('wm', 'attributes', '.', '-topmost', '1')
                root.withdraw();
                InputFold = tk.filedialog.askdirectory(parent=root,initialdir = r'E:\Users\Davide_Marchegiani\SoilMoisture')
        if OutputFold is None:
                OutputFold = InputFold + '_STACK'
        os.makedirs(OutputFold,exist_ok=True)
        
        print('Begin algorithm...')
        sys.stdout.flush()
        
        st=time.time()
          
        files_list=glob.glob(InputFold+'\*.dim')
        files_list=sorted(files_list,key=lambda x: os.path.split(x)[1][4:])
        
        #Initialize snappy
        HashMap = jpy.get_type('java.util.HashMap')
        #Get snappy operators    
        GPF.getDefaultInstance().getOperatorSpiRegistry().loadOperatorSpis()
        
        ind1= [m.start() for m in re.finditer('_',os.path.split(files_list[0])[1])]
        ind2= [m.start() for m in re.finditer('_',os.path.split(files_list[-1])[1])]
        #Differentiate between S1 and S2
        if os.path.split(files_list[0])[1][1] is '1':
                date_start = os.path.split(files_list[0])[1][ind1[3]+1:ind1[3]+9]
                date_end = os.path.split(files_list[-1])[1][ind2[3]+1:ind2[3]+9]
                
        elif os.path.split(files_list[0])[1][1] is '2':
                date_start = os.path.split(files_list[0])[1][ind1[1]+1:ind1[1]+9]
                date_end = os.path.split(files_list[-1])[1][ind2[1]+1:ind2[1]+9]                
        #==============================================================================
        # CREATE NUMBER OF PRODUCTS EQUAL TO len(bnames)
        #==============================================================================
        source = ProductIO.readProduct(os.path.join(InputFold,files_list[0]))
        if bnames is None:
                bnames = source.getBandNames()
        elif type(bnames) is str:
                bnames=[bnames]
        
        width=source.getSceneRasterWidth()
        height=source.getSceneRasterHeight()
        writer = ProductIO.getProductWriter('BEAM-DIMAP')
        
        products = jpy.array('org.esa.snap.core.datamodel.Product',len(bnames))
        for i in range(len(bnames)):
                products[i] = Product(bnames[i]+'_Stack_'+date_start+'_'+date_end, source.getProductType(), width, height)
                snappy.ProductUtils.copyGeoCoding(source, products[i])
                products[i].setProductWriter(writer)
        #==============================================================================
        # LOOP OVER THE FILES
        #==============================================================================
        print('Stacking the bands...')
        sys.stdout.flush()
        for file in files_list:
                ind= [m.start() for m in re.finditer('_',os.path.split(file)[1])]
                #Differentiate between S1 and S2
                if os.path.split(files_list[0])[1][1] is '1':
                        date = os.path.split(file)[1][ind[3]+1:ind[3]+9]
                elif os.path.split(files_list[0])[1][1] is '2':
                        date = os.path.split(file)[1][ind[1]+1:ind[1]+9]     
                
                # OPEN FILES
                source = ProductIO.readProduct(file)
                # ADD THE SELECTED BANDS TO THE PRODUCTS CREATED
                for i in range(len(bnames)):
                        ProductUtils.copyBand(bnames[i],source,bnames[i]+'_'+date,products[i],True)
        #==============================================================================
        # ADD LAT and LON BANDS TO THE PRODUCTS
        #==============================================================================
#        # THIS PART HAS BEEN COMMENTED BECAUSE IT SEEMS NOT TO WORK AS IT SHOULD. 
#        # ADDING THE LAT AND LON BANDS FROM GUI BANDMATHS WORKS FINE.
#        
#        
#        print('Adding LON and LAT bands...')
#        sys.stdout.flush()
#        BandDescriptor = jpy.get_type('org.esa.snap.core.gpf.common.BandMathsOp$BandDescriptor')
#        
#        #Latitude Band
#        lat = BandDescriptor()
#        lat.name = 'Lat'
#        lat.type = 'float64'
#        lat.expression = 'LAT'
#        #Longitude Band
#        lon = BandDescriptor()
#        lon.name = 'Lon'
#        lon.type = 'float64'
#        lon.expression = 'LON'
#        
#        newBands = jpy.array('org.esa.snap.core.gpf.common.BandMathsOp$BandDescriptor', 2)
#        newBands[0] = lat
#        newBands[1] = lon
#        #Compute BANDMATHS Operator        
#        parameters = HashMap()
#        parameters.put('targetBands', newBands)
#        prod1 = GPF.createProduct('BandMaths', parameters, source)
#        #Get new bands and add to the Products created
#        for prod in products:
#                ProductUtils.copyBand('Latitude',prod1,prod,True)
#                ProductUtils.copyBand('Longitude',prod1,prod,True)
        #==============================================================================
        # WRITE PRODUCTS TO FILE 
        #==============================================================================
        for prod in products:
                print('Writing product "{}"...'.format(prod.getName()))
                sys.stdout.flush()
                time.sleep(.1)
                ProductIO.writeProduct(prod,os.path.join(OutputFold,os.path.split(files_list[-1])[1][1:ind[3]]+'_'+prod.getName()), 'BEAM-DIMAP')
                print('Product "{}" wrote! - t = {:.2f}s'.format(prod.getName(),time.time()-st))
        print('Done! - t = {:.2f}s'.format(time.time()-st))

If you can find any problems related to time consume let me know, thank you!

A post was merged into an existing topic: Get latitude/longitude values in python operator

# Terrain correction
per=HashMap()
per.put('sourceBands', 'Sigma0_VH,Sigma0_VV')
per.put('demResamplingMethod','BILINEAR_INTERPOLATION')
per.put('imgResamplingMethod','BILINEAR_INTERPOLATION')
per.put('demName','SRTM 3Sec')
per.put('pixelSpacingInMeter', 10.0)
per.put('nodataValueAtSea',False)
per.put('saveIncidenceAngleFromEllipsoid',True)
per.put('saveLocalIncidenceAngle',True)
Ter_cor=GPF.createProduct('Terrain-Correction',per,sig_im)
list(Ter_cor.getBandNames())
Out:  ['Sigma0_VH',
'Sigma0_VV',
'localIncidenceAngle',
'incidenceAngleFromEllipsoid']
#export
ProductIO.writeProduct(Ter_cor,r'E:\snappy_sen1GRD\af_cor.dim',  'BEAM-DIMAP')

Q1: After write first 2 bands snappy stopped writing next 2 bands. I wait for 1 hour then I stopped the program. what is wrong here?
Q2: How I can add Externel DEM in Terrain-Correction?
Q3: How to crop image using shapefile in this interface and how I can add the shapefile?
Q4: Can I use vRAM in snappy (Windows Paging) for fast processing?
Q5: How I can export a single band?

Please help me. I am not a pro in coding. Learning everything.
Here Is the output of VV band after export. It is unable to write full image.

hello @Jeongju can you please let me know how to read 2 band at a time and with the help of math i changed the array and how to save it ?

@marpet @Jeongju
I got the below error for same code
RuntimeError: java.lang.IllegalStateException: no product reader for band '_Enhanced'

I don’t know what you are doing. Are you trying to read data before the product has been written?
In the script above a band is created with the name “_Enhanced” it is does not have a reader attached. Therefore, the exception is reasonable.

Thankyou very much sir @marpet i solved this
but i am phasing new error

Traceback (most recent call last):
  File "D:\Product_Team\Final_with_soil_moisture.py", line 163, in <module>
    arr_VH = VH.readPixels(0,0,width,height,dataline)
RuntimeError: org.esa.snap.core.gpf.OperatorException: Cannot construct DataBuffer.

when i am trying to save output with subset it was working fine but in case of full image i got this kind of error.

by the way i have 32GB RAM but still this error comes.

did you make sure that SNAP (or snappy) is able to make full use of your RAM?

I have set java_max_mem: 26G