Snappy not freeing memory

Ciaran_Evans · March 13, 2018, 7:34pm

Hi there,

I’ve developed a python api for generating flood mapping with snappy. When I call it a few times I occasionally get a large stack trace as follows:

    java.lang.NullPointerException                                                                                           
        at com.sun.media.jai.util.SunCachedTile.<init>(SunCachedTile.java:80)
        at com.sun.media.jai.util.SunTileCache.add(SunTileCache.java:257)
        at javax.media.jai.OpImage.addTileToCache(OpImage.java:1087)
        at javax.media.jai.OpImage.getTile(OpImage.java:1142)
        at javax.media.jai.PlanarImage.cobbleInt(PlanarImage.java:3153)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2178)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.calibration.gpf.calibrators.Sentinel1Calibrator.computeTile(Sentinel1Calibrator.java:533)
        at org.esa.s1tbx.calibration.gpf.CalibrationOp.computeTile(CalibrationOp.java:190)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.datamodel.Band.readRasterData(Band.java:311)
        at org.esa.snap.core.dataio.ProductSubsetBuilder.readBandRasterDataRegion(ProductSubsetBuilder.java:328)
        at org.esa.snap.core.dataio.ProductSubsetBuilder.readBandRasterDataImpl(ProductSubsetBuilder.java:297)
        at org.esa.snap.core.dataio.AbstractProductReader.readBandRasterData(AbstractProductReader.java:250)
        at org.esa.snap.core.gpf.common.SubsetOp.computeTile(SubsetOp.java:270)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilters.Lee.performFiltering(Lee.java:116)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilters.Lee.computeTile(Lee.java:82)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilterOp.computeTile(SpeckleFilterOp.java:299)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)                                                            
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.snap.core.gpf.common.BandMathsOp.fillSymbolWithData(BandMathsOp.java:355)
        at org.esa.snap.core.gpf.common.BandMathsOp.computeTile(BandMathsOp.java:322)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.getPixelValue(RangeDopplerGeocodingOp.java:1255)
        at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.computeTileStack(RangeDopplerGeocodingOp.java:1065)
        at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeRect(OperatorImageTileStack.java:116)
        at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeTile(OperatorImageTileStack.java:85)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.datamodel.Band.writeRasterDataFully(Band.java:384)
        at org.esa.snap.core.dataio.ProductIO.writeAllBands(ProductIO.java:448)
        at org.esa.snap.core.dataio.ProductIO.writeProduct(ProductIO.java:401)
        at org.esa.snap.core.dataio.ProductIO.writeProduct(ProductIO.java:330)
java.lang.NullPointerException
        at com.sun.media.jai.util.SunCachedTile.<init>(SunCachedTile.java:80)
        at com.sun.media.jai.util.SunTileCache.add(SunTileCache.java:257)
        at javax.media.jai.OpImage.addTileToCache(OpImage.java:1087)
        at javax.media.jai.OpImage.getTile(OpImage.java:1142)
        at javax.media.jai.PlanarImage.cobbleInt(PlanarImage.java:3153)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2178)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.calibration.gpf.calibrators.Sentinel1Calibrator.computeTile(Sentinel1Calibrator.java:533)       
        at org.esa.s1tbx.calibration.gpf.CalibrationOp.computeTile(CalibrationOp.java:190)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.datamodel.Band.readRasterData(Band.java:311)
        at org.esa.snap.core.dataio.ProductSubsetBuilder.readBandRasterDataRegion(ProductSubsetBuilder.java:328)
        at org.esa.snap.core.dataio.ProductSubsetBuilder.readBandRasterDataImpl(ProductSubsetBuilder.java:297)
        at org.esa.snap.core.dataio.AbstractProductReader.readBandRasterData(AbstractProductReader.java:250)
        at org.esa.snap.core.gpf.common.SubsetOp.computeTile(SubsetOp.java:270)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilters.Lee.performFiltering(Lee.java:116)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilters.Lee.computeTile(Lee.java:82)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilterOp.computeTile(SpeckleFilterOp.java:299)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.snap.core.gpf.common.BandMathsOp.fillSymbolWithData(BandMathsOp.java:355)
        at org.esa.snap.core.gpf.common.BandMathsOp.computeTile(BandMathsOp.java:322)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)                                                            
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.getPixelValue(RangeDopplerGeocodingOp.java:1255)
        at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.computeTileStack(RangeDopplerGeocodingOp.java:1065)
        at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeRect(OperatorImageTileStack.java:116)
        at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeTile(OperatorImageTileStack.java:85)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.datamodel.Band.writeRasterDataFully(Band.java:384)
        at org.esa.snap.core.dataio.ProductIO.writeAllBands(ProductIO.java:448)
        at org.esa.snap.core.dataio.ProductIO.writeProduct(ProductIO.java:401)
        at org.esa.snap.core.dataio.ProductIO.writeProduct(ProductIO.java:330)
java.lang.NullPointerException
        at com.sun.media.jai.util.SunCachedTile.<init>(SunCachedTile.java:80)
        at com.sun.media.jai.util.SunTileCache.add(SunTileCache.java:257)
        at javax.media.jai.OpImage.addTileToCache(OpImage.java:1087)
        at javax.media.jai.OpImage.getTile(OpImage.java:1142)
        at javax.media.jai.PlanarImage.cobbleInt(PlanarImage.java:3153)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2178)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.calibration.gpf.calibrators.Sentinel1Calibrator.computeTile(Sentinel1Calibrator.java:533)
        at org.esa.s1tbx.calibration.gpf.CalibrationOp.computeTile(CalibrationOp.java:190)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.datamodel.Band.readRasterData(Band.java:311)
        at org.esa.snap.core.dataio.ProductSubsetBuilder.readBandRasterDataRegion(ProductSubsetBuilder.java:328)
        at org.esa.snap.core.dataio.ProductSubsetBuilder.readBandRasterDataImpl(ProductSubsetBuilder.java:297)           
        at org.esa.snap.core.dataio.AbstractProductReader.readBandRasterData(AbstractProductReader.java:250)
        at org.esa.snap.core.gpf.common.SubsetOp.computeTile(SubsetOp.java:270)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilters.Lee.performFiltering(Lee.java:116)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilters.Lee.computeTile(Lee.java:82)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilterOp.computeTile(SpeckleFilterOp.java:299)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.snap.core.gpf.common.BandMathsOp.fillSymbolWithData(BandMathsOp.java:355)
        at org.esa.snap.core.gpf.common.BandMathsOp.computeTile(BandMathsOp.java:322)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.getPixelValue(RangeDopplerGeocodingOp.java:1255)
        at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.computeTileStack(RangeDopplerGeocodingOp.java:1065)
        at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeRect(OperatorImageTileStack.java:116)
        at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeTile(OperatorImageTileStack.java:85)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)                               
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.datamodel.Band.writeRasterDataFully(Band.java:384)
        at org.esa.snap.core.dataio.ProductIO.writeAllBands(ProductIO.java:448)
        at org.esa.snap.core.dataio.ProductIO.writeProduct(ProductIO.java:401)
        at org.esa.snap.core.dataio.ProductIO.writeProduct(ProductIO.java:330)
java.lang.NullPointerException
        at com.sun.media.jai.util.SunCachedTile.<init>(SunCachedTile.java:80)
        at com.sun.media.jai.util.SunTileCache.add(SunTileCache.java:257)
        at javax.media.jai.OpImage.addTileToCache(OpImage.java:1087)
        at javax.media.jai.OpImage.getTile(OpImage.java:1142)
        at javax.media.jai.PlanarImage.cobbleInt(PlanarImage.java:3153)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2178)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.calibration.gpf.calibrators.Sentinel1Calibrator.computeTile(Sentinel1Calibrator.java:533)
        at org.esa.s1tbx.calibration.gpf.CalibrationOp.computeTile(CalibrationOp.java:190)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.datamodel.Band.readRasterData(Band.java:311)
        at org.esa.snap.core.dataio.ProductSubsetBuilder.readBandRasterDataRegion(ProductSubsetBuilder.java:328)
        at org.esa.snap.core.dataio.ProductSubsetBuilder.readBandRasterDataImpl(ProductSubsetBuilder.java:297)
        at org.esa.snap.core.dataio.AbstractProductReader.readBandRasterData(AbstractProductReader.java:250)
        at org.esa.snap.core.gpf.common.SubsetOp.computeTile(SubsetOp.java:270)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)                         
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilters.Lee.performFiltering(Lee.java:116)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilters.Lee.computeTile(Lee.java:82)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilterOp.computeTile(SpeckleFilterOp.java:299)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.snap.core.gpf.common.BandMathsOp.fillSymbolWithData(BandMathsOp.java:355)
        at org.esa.snap.core.gpf.common.BandMathsOp.computeTile(BandMathsOp.java:322)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.getPixelValue(RangeDopplerGeocodingOp.java:1255)
        at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.computeTileStack(RangeDopplerGeocodingOp.java:1065)
        at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeRect(OperatorImageTileStack.java:116)
        at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeTile(OperatorImageTileStack.java:85)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.datamodel.Band.writeRasterDataFully(Band.java:384)
        at org.esa.snap.core.dataio.ProductIO.writeAllBands(ProductIO.java:448)
        at org.esa.snap.core.dataio.ProductIO.writeProduct(ProductIO.java:401)
        at org.esa.snap.core.dataio.ProductIO.writeProduct(ProductIO.java:330)
java.lang.NullPointerException
        at com.sun.media.jai.util.SunCachedTile.<init>(SunCachedTile.java:80)
        at com.sun.media.jai.util.SunTileCache.add(SunTileCache.java:257)
        at javax.media.jai.OpImage.addTileToCache(OpImage.java:1087)                                                      
        at javax.media.jai.OpImage.getTile(OpImage.java:1142)
        at javax.media.jai.PlanarImage.cobbleInt(PlanarImage.java:3153)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2178)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.calibration.gpf.calibrators.Sentinel1Calibrator.computeTile(Sentinel1Calibrator.java:533)
        at org.esa.s1tbx.calibration.gpf.CalibrationOp.computeTile(CalibrationOp.java:190)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.datamodel.Band.readRasterData(Band.java:311)
        at org.esa.snap.core.dataio.ProductSubsetBuilder.readBandRasterDataRegion(ProductSubsetBuilder.java:328)
        at org.esa.snap.core.dataio.ProductSubsetBuilder.readBandRasterDataImpl(ProductSubsetBuilder.java:297)
        at org.esa.snap.core.dataio.AbstractProductReader.readBandRasterData(AbstractProductReader.java:250)
        at org.esa.snap.core.gpf.common.SubsetOp.computeTile(SubsetOp.java:270)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilters.Lee.performFiltering(Lee.java:116)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilters.Lee.computeTile(Lee.java:82)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilterOp.computeTile(SpeckleFilterOp.java:299)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.snap.core.gpf.common.BandMathsOp.fillSymbolWithData(BandMathsOp.java:355)
        at org.esa.snap.core.gpf.common.BandMathsOp.computeTile(BandMathsOp.java:322)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.getPixelValue(RangeDopplerGeocodingOp.java:1255)
        at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.computeTileStack(RangeDopplerGeocodingOp.java:1065)
        at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeRect(OperatorImageTileStack.java:116)
        at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeTile(OperatorImageTileStack.java:85)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.datamodel.Band.writeRasterDataFully(Band.java:384)
        at org.esa.snap.core.dataio.ProductIO.writeAllBands(ProductIO.java:448)
        at org.esa.snap.core.dataio.ProductIO.writeProduct(ProductIO.java:401)
        at org.esa.snap.core.dataio.ProductIO.writeProduct(ProductIO.java:330)

Apologies for the huge paste job, but I was wondering whats causing this? It seems to work quite quickly with the caching so if I use the same WKT to subset on and use the same image, the process is done in seconds.

Just seems strange that the errors only appear after a few calls. Any help is appreciated!

Thanks,

Ciaran

Ciaran_Evans · March 13, 2018, 10:01pm

Further on to this, it seems to be an issue with caching after multiple calls.

The process seems to hang but then does eventually write out the .tiff, yet the error still occurs.

Seems caching only works a few times? Then I effectively have to process the whole image again without it…

Ciaran_Evans · March 18, 2018, 6:26pm

Bumping this with some more investigating.

I bumped my VM up to 8GB and also bumped up the java_max_mem property in snappy.ini to 8G

I’m processing some S1 imagery and subsetting it to POLYGON ((-4.136685132980347 50.36611236042063, -4.12860631942749 50.36611236042063, -4.12860631942749 50.37036895441764, -4.136685132980347 50.37036895441764, -4.136685132980347 50.36611236042063)) so its very small in terms of a S1 image.

Running the process once I get python hogging 16% of the VMs memory, making several calls it always idled at more memory:

16%, 28%, 42%, 52%, 81%, 81%, 95%, 95%, 90%, crashed.

I’m assuming this is to do with the Java side because memory just isn’t being freed. I’m removing all files in /tmp every time I run the process too so. Is there anyway to stop this continuous buildup of memory till eventual crashing?

Many thanks,

Ciaran

Ciaran_Evans · March 19, 2018, 11:52am

Further to this, I’ve noticed posts mentioning forcing garbage collection via something such as jpy.get_type('java.lang.System').gc() and this doesn’t seem to do anything. It actually seems to increase the memory usage…

Ciaran_Evans · March 21, 2018, 12:30pm

Wondered if anyone has any suggestions RE this? Renders my pipeline useless after a few calls.

Ciaran_Evans · March 22, 2018, 1:32pm

@marpet sorry to pester! Do you know of any fixes/workarounds to this?

mdkwant · March 22, 2018, 1:57pm

Snappy is known to have memory issues, especially with clearing the memory after running a script. What might be worth a try is running your python script from Command Window, instead of Spyder (i particularly experienced problems with Spyder).

Ciaran_Evans · March 22, 2018, 2:05pm

The script is being running through a Flask application http://flask.pocoo.org so it’s not a issue with Spyder.

I’m hosting this as an api to call so having it effectively bomb out after a few calls isn’t handy at all, so far my only option is to write a cron job to kill the process and restart it, but that’s basically making the api useless as it could cut out when it’s running a job.

Frustrating that trying to force garbage collection doesn’t seem to help.

marpet · April 15, 2018, 9:39am

No, I have no clue. Sorry. I also don’t see meaningful improvements coming very soon.
I still think that the major cause is the S1 data, respectively the S1 reader. But I’m not sure.

Ciaran_Evans · April 16, 2018, 8:08pm

That’s unfortunate to hear, I know it’s affecting a few users.

@seb is also experiencing issues with this. It makes creating apps hard/unmanageable as they eventually die from running out of memory.

Wonder if there’s anyone else who has suggestions for this?

jmigueldelgado · April 26, 2018, 7:24am

Hi Ciaran,

Garbage collection did not help me either. Maybe something related to this? I had a similar memory issues because I was invoking jpy.get_type() inside a loop. It is better to move out of the loop. It solved the issue for me.
Cheers
jose

Ciaran_Evans · April 26, 2018, 12:06pm

Thanks for the suggestion!

I managed to solve my issues with:

I’ll give yours a go sometime to see if it saves me doing my workaround!

pedroLanz · August 9, 2018, 2:03pm

at least, it closes the file and enables windows preview

lumaro · November 6, 2018, 10:02am

Hi,

Thank you very much Ciaran and Marco.

I have the same problem. The only effective solution that worked in my case was the one of @Ciaran_Evans (launching and independent process).

@marpet is there any open ticket or similar about the issue?

harmel · November 6, 2018, 1:47pm

Same issue here, especially when using the S2sampler

marpet · November 7, 2018, 8:50am

No there is no special ticket for memory issue in snappy, but there are some others
https://senbox.atlassian.net/browse/SNAP-869
https://senbox.atlassian.net/browse/SITBX-562
https://senbox.atlassian.net/browse/SITBX-482

I’ve created one now. (SNAP-960). For me it is not yet clear where the memory issues comes from. Maybe it is within SNAP, maybe in jpy or in snappy. This needs to be investigated.
However, I can tell that the further development of SNAP and the Toolboxes will start again soon. And then we will tackle such memory and performance issues. Finally…
But till the fixes will be released it still might take a few month.

Ciaran_Evans · November 13, 2018, 2:37pm

@lumaro - glad it fixed your issue (for now)

Thanks @marpet for raising the issue! Hopefully the workaround can be useful for people at the moment, I know here at work our Remote Sensing folk have hit the issue too.

@harmel I’d recommend giving the workaround ago, it’s by no means pretty but it allows for bulk processing. Give me a shout if you need a hand!

harmel · November 19, 2018, 9:28am

Thanks a lot @Ciaran_Evans .
I noticed that image data are loaded by chunk of 512 pixel rows. So for me the work-around is to run the process for each 512-row chunk independently.
Cheers,
Tristan

harmel · September 10, 2019, 2:01pm

Any news about memory leaks and/or possibilities of freeing memory from snappy with snap V7?

MarkWilliamMatthews · June 10, 2020, 1:22pm

Hi Marco, just providing some diagnostic info that I hope can help you solve the issue. In my experience, the problem occurs with GPF.createProduct(). When using ProductIO.readProduct and ProductIO.writeProduct the memory issues do not appear provided you use the .dispose() method on all products you read. After using product = GPF.createProduct() the product.dispose() method is ineffective - the memory is not released. So perhaps the problem has to do with using the dispose method on the products created using GPF. Hope it helps!