Computing filtered bands

wasdi · December 15, 2017, 9:02am

hi to everyone. In our application we must programmatically compute a FilterBand and save it as geotiff (using java)

at the moment i’m trying to do it with the following code, but the result is very very slow… it takes hours, while exporting the geotiff from snap desktop is faster (about 10 seconds to create the image showed and about 40 seconds to export the computed band in geotiff format)

File file = new File("S1A_IW_GRDH_1SDV_20171128T054335_20171128T054400_019461_02104F_DFC1.zip");

Product product = ProductIO.readProduct(file);
String bandName = "Amplitude_VH";

RasterDataNode node = product.getRasterDataNode(bandName); 
Filter filter = StandardFilters.SMOOTHING_FILTERS[0];

String newBandName = bandName + "_" + filter.getShorthand();
File fileOut = new File(newBandName + ".tif");


FilterBand computedBand = getFilterBand(node, newBandName, filter, 10);
System.out.println("filter band created");

MultiLevelImage bandImage = computedBand.getSourceImage();

System.out.println("surce image created");

// Get TIFF Metadata
GeoTIFFMetadata metadata = ProductUtils.createGeoTIFFMetadata(product);

System.out.println("geotiff metadata created");

GeoTIFF.writeImage(bandImage, fileOut, metadata);

System.out.println("geotiff created");

where getFilterBand is the following method (copied from the snap class org.esa.snap.rcp.imgfilter.FilteredBandAction)

private static FilterBand getFilterBand(RasterDataNode sourceRaster, String bandName, Filter filter, int iterationCount) {
    FilterBand targetBand;

    if (filter.getOperation() == Filter.Operation.CONVOLVE) {
        targetBand = new ConvolutionFilterBand(bandName, sourceRaster, getKernel(filter), iterationCount);
        if (sourceRaster instanceof Band) {
            ProductUtils.copySpectralBandProperties((Band) sourceRaster, targetBand);
        }
    } else {
        GeneralFilterBand.OpType opType = getOpType(filter.getOperation());
        targetBand = new GeneralFilterBand(bandName, sourceRaster, opType, getKernel(filter), iterationCount);
        if (sourceRaster instanceof Band) {
            ProductUtils.copySpectralBandProperties((Band) sourceRaster, targetBand);
        }
    }

    targetBand.setDescription(String.format("Filter '%s' (=%s) applied to '%s'", filter.getName(), filter.getOperation(), sourceRaster.getName()));
    if (sourceRaster instanceof Band) {
        ProductUtils.copySpectralBandProperties((Band) sourceRaster, targetBand);
    }
    
    ProductUtils.copyImageGeometry(sourceRaster, targetBand, false);
    targetBand.fireProductNodeDataChanged();
    
    return targetBand;
}

is this the correct way to do that?

thanks in advance

marpet · December 15, 2017, 10:19am

Actually, your code looks quite well. Maybe your memory settings are too low.
I think if you use the GeoTiff.writeImage() method the whole image is loaded into memory at once.
You can try to create a new product, add the filtered band to the product and then write the product.
Use ProductIO.writeProduct() for this. Then the data is written tile-wise.

wasdi · December 18, 2017, 9:46am

after changing my code in this way

    FilterBand computedBand = getFilterBand(node, newBandName, filter, 10);
	Band realBand = new Band(newBandName, computedBand.getDataType(), computedBand.getRasterWidth(), computedBand.getRasterHeight());
	realBand.setSourceImage(computedBand.getSourceImage());
	
	System.out.println("filter band created");
	
	// Get TIFF Metadata
	System.out.println("create product");
	Product outProduct = new Product(newBandName, "GEOTIFF");
	ProductUtils.copyGeoCoding(product, outProduct);
	outProduct.addBand(realBand);
	System.out.println("create geotiff");
	ProductIO.writeProduct(outProduct, fileOut, "GEOTIFF", true, new ProgressMonitor() {
		
		@Override
		public void worked(int work) {
			System.out.println("worked: " + work);
		}
		
		@Override
		public void setTaskName(String taskName) {
			System.out.println("task name: " + taskName);				
		}
		
		@Override
		public void setSubTaskName(String subTaskName) {
			System.out.println("sub task name: " + subTaskName);
		}
		
		@Override
		public void setCanceled(boolean canceled) {
			System.out.println("cancelled: " + canceled);
		}
		
		@Override
		public boolean isCanceled() {
			return false;
		}
		
		@Override
		public void internalWorked(double work) {
			System.out.println("internal worked: " + work);
		}
		
		@Override
		public void done() {
			System.out.println("done");
		}
		
		@Override
		public void beginTask(String taskName, int totalWork) {
			System.out.println("begin task " + taskName + ". work: " + totalWork);
		}
	} );

	System.out.println("geotiff created");

and the java parameters in this way

-Xmx6144m -XX:+AggressiveOpts -Xverify:none -Dsun.java2d.noddraw=true -Dsun.awt.nopixfmt=true -Dsun.java2d.dpiaware=false -Dsnap.jai.tileCacheSize=1024

the result is the same… here is the output (process manually stopped after 10 minutes):

WARNING: org.esa.snap.core.util.ServiceLoader: org.esa.snap.core.dataio.ProductReaderPlugIn: Provider org.esa.snap.dataio.arcbin.ArcBinGridReaderPlugIn not found
WARNING: org.esa.snap.core.util.ServiceLoader: org.esa.snap.core.dataio.ProductReaderPlugIn: Provider org.esa.snap.binning.reader.BinnedProductReaderPlugin not found
WARNING: org.esa.snap.core.util.ServiceLoader: org.esa.snap.core.dataio.ProductReaderPlugIn: Provider org.esa.snap.csv.dataio.reader.CsvProductReaderPlugIn not found
WARNING: org.esa.snap.core.util.ServiceLoader: org.esa.snap.core.dataio.ProductReaderPlugIn: Provider org.esa.snap.dataio.getasse30.GETASSE30ReaderPlugIn not found
WARNING: org.esa.snap.core.util.ServiceLoader: org.esa.snap.core.dataio.ProductReaderPlugIn: Provider org.esa.snap.dataio.pgx.PgxProductReaderPlugIn not found
WARNING: org.esa.snap.core.util.ServiceLoader: org.esa.snap.core.dataio.ProductReaderPlugIn: Provider org.esa.snap.dataio.rtp.RawTiledPyramidsProductCodecSpi not found
WARNING: org.esa.snap.core.util.ServiceLoader: org.esa.snap.core.dataio.ProductReaderPlugIn: Provider org.esa.s2tbx.dataio.deimos.DeimosProductReaderPlugin not found
WARNING: org.esa.snap.core.util.ServiceLoader: org.esa.snap.core.dataio.ProductWriterPlugIn: Provider org.esa.snap.csv.dataio.writer.CsvProductWriterPlugIn not found
WARNING: org.esa.snap.core.util.ServiceLoader: org.esa.snap.core.dataio.ProductWriterPlugIn: Provider org.esa.snap.dataio.hdf5.Hdf5ProductWriterPlugIn not found
WARNING: org.esa.snap.core.util.ServiceLoader: org.esa.snap.core.dataio.ProductWriterPlugIn: Provider org.esa.snap.dataio.rtp.RawTiledPyramidsProductCodecSpi not found
SLF4J: The requested version 1.6 by your slf4j binding is not compatible with [1.5.5, 1.5.6, 1.5.7, 1.5.8, 1.5.9, 1.5.10, 1.5.11]
SLF4J: See http://www.slf4j.org/codes.html#version_mismatch for further details.
log4j:WARN No appenders could be found for logger (hsqldb.db.HSQLDB4AD417742A.ENGINE).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
filter band created
create product
create geotiff
begin task Writing bands of product 'Amplitude_VH_am3'.... work: 1
sub task name: Writing band 'Amplitude_VH_am3'

I don’t know what to do …thanks in advance for any help

marpet · January 8, 2018, 10:11am

Maybe you need to wait longer than ten minutes. You can also increase the tileCacheSize to 3000. This might improve the performance.