Snappy not freeing memory


#1

Hi there,

I’ve developed a python api for generating flood mapping with snappy. When I call it a few times I occasionally get a large stack trace as follows:

    java.lang.NullPointerException                                                                                           
        at com.sun.media.jai.util.SunCachedTile.<init>(SunCachedTile.java:80)
        at com.sun.media.jai.util.SunTileCache.add(SunTileCache.java:257)
        at javax.media.jai.OpImage.addTileToCache(OpImage.java:1087)
        at javax.media.jai.OpImage.getTile(OpImage.java:1142)
        at javax.media.jai.PlanarImage.cobbleInt(PlanarImage.java:3153)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2178)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.calibration.gpf.calibrators.Sentinel1Calibrator.computeTile(Sentinel1Calibrator.java:533)
        at org.esa.s1tbx.calibration.gpf.CalibrationOp.computeTile(CalibrationOp.java:190)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.datamodel.Band.readRasterData(Band.java:311)
        at org.esa.snap.core.dataio.ProductSubsetBuilder.readBandRasterDataRegion(ProductSubsetBuilder.java:328)
        at org.esa.snap.core.dataio.ProductSubsetBuilder.readBandRasterDataImpl(ProductSubsetBuilder.java:297)
        at org.esa.snap.core.dataio.AbstractProductReader.readBandRasterData(AbstractProductReader.java:250)
        at org.esa.snap.core.gpf.common.SubsetOp.computeTile(SubsetOp.java:270)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilters.Lee.performFiltering(Lee.java:116)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilters.Lee.computeTile(Lee.java:82)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilterOp.computeTile(SpeckleFilterOp.java:299)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)                                                            
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.snap.core.gpf.common.BandMathsOp.fillSymbolWithData(BandMathsOp.java:355)
        at org.esa.snap.core.gpf.common.BandMathsOp.computeTile(BandMathsOp.java:322)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.getPixelValue(RangeDopplerGeocodingOp.java:1255)
        at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.computeTileStack(RangeDopplerGeocodingOp.java:1065)
        at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeRect(OperatorImageTileStack.java:116)
        at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeTile(OperatorImageTileStack.java:85)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.datamodel.Band.writeRasterDataFully(Band.java:384)
        at org.esa.snap.core.dataio.ProductIO.writeAllBands(ProductIO.java:448)
        at org.esa.snap.core.dataio.ProductIO.writeProduct(ProductIO.java:401)
        at org.esa.snap.core.dataio.ProductIO.writeProduct(ProductIO.java:330)
java.lang.NullPointerException
        at com.sun.media.jai.util.SunCachedTile.<init>(SunCachedTile.java:80)
        at com.sun.media.jai.util.SunTileCache.add(SunTileCache.java:257)
        at javax.media.jai.OpImage.addTileToCache(OpImage.java:1087)
        at javax.media.jai.OpImage.getTile(OpImage.java:1142)
        at javax.media.jai.PlanarImage.cobbleInt(PlanarImage.java:3153)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2178)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.calibration.gpf.calibrators.Sentinel1Calibrator.computeTile(Sentinel1Calibrator.java:533)       
        at org.esa.s1tbx.calibration.gpf.CalibrationOp.computeTile(CalibrationOp.java:190)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.datamodel.Band.readRasterData(Band.java:311)
        at org.esa.snap.core.dataio.ProductSubsetBuilder.readBandRasterDataRegion(ProductSubsetBuilder.java:328)
        at org.esa.snap.core.dataio.ProductSubsetBuilder.readBandRasterDataImpl(ProductSubsetBuilder.java:297)
        at org.esa.snap.core.dataio.AbstractProductReader.readBandRasterData(AbstractProductReader.java:250)
        at org.esa.snap.core.gpf.common.SubsetOp.computeTile(SubsetOp.java:270)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilters.Lee.performFiltering(Lee.java:116)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilters.Lee.computeTile(Lee.java:82)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilterOp.computeTile(SpeckleFilterOp.java:299)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.snap.core.gpf.common.BandMathsOp.fillSymbolWithData(BandMathsOp.java:355)
        at org.esa.snap.core.gpf.common.BandMathsOp.computeTile(BandMathsOp.java:322)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)                                                            
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.getPixelValue(RangeDopplerGeocodingOp.java:1255)
        at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.computeTileStack(RangeDopplerGeocodingOp.java:1065)
        at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeRect(OperatorImageTileStack.java:116)
        at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeTile(OperatorImageTileStack.java:85)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.datamodel.Band.writeRasterDataFully(Band.java:384)
        at org.esa.snap.core.dataio.ProductIO.writeAllBands(ProductIO.java:448)
        at org.esa.snap.core.dataio.ProductIO.writeProduct(ProductIO.java:401)
        at org.esa.snap.core.dataio.ProductIO.writeProduct(ProductIO.java:330)
java.lang.NullPointerException
        at com.sun.media.jai.util.SunCachedTile.<init>(SunCachedTile.java:80)
        at com.sun.media.jai.util.SunTileCache.add(SunTileCache.java:257)
        at javax.media.jai.OpImage.addTileToCache(OpImage.java:1087)
        at javax.media.jai.OpImage.getTile(OpImage.java:1142)
        at javax.media.jai.PlanarImage.cobbleInt(PlanarImage.java:3153)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2178)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.calibration.gpf.calibrators.Sentinel1Calibrator.computeTile(Sentinel1Calibrator.java:533)
        at org.esa.s1tbx.calibration.gpf.CalibrationOp.computeTile(CalibrationOp.java:190)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.datamodel.Band.readRasterData(Band.java:311)
        at org.esa.snap.core.dataio.ProductSubsetBuilder.readBandRasterDataRegion(ProductSubsetBuilder.java:328)
        at org.esa.snap.core.dataio.ProductSubsetBuilder.readBandRasterDataImpl(ProductSubsetBuilder.java:297)           
        at org.esa.snap.core.dataio.AbstractProductReader.readBandRasterData(AbstractProductReader.java:250)
        at org.esa.snap.core.gpf.common.SubsetOp.computeTile(SubsetOp.java:270)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilters.Lee.performFiltering(Lee.java:116)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilters.Lee.computeTile(Lee.java:82)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilterOp.computeTile(SpeckleFilterOp.java:299)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.snap.core.gpf.common.BandMathsOp.fillSymbolWithData(BandMathsOp.java:355)
        at org.esa.snap.core.gpf.common.BandMathsOp.computeTile(BandMathsOp.java:322)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.getPixelValue(RangeDopplerGeocodingOp.java:1255)
        at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.computeTileStack(RangeDopplerGeocodingOp.java:1065)
        at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeRect(OperatorImageTileStack.java:116)
        at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeTile(OperatorImageTileStack.java:85)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)                               
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.datamodel.Band.writeRasterDataFully(Band.java:384)
        at org.esa.snap.core.dataio.ProductIO.writeAllBands(ProductIO.java:448)
        at org.esa.snap.core.dataio.ProductIO.writeProduct(ProductIO.java:401)
        at org.esa.snap.core.dataio.ProductIO.writeProduct(ProductIO.java:330)
java.lang.NullPointerException
        at com.sun.media.jai.util.SunCachedTile.<init>(SunCachedTile.java:80)
        at com.sun.media.jai.util.SunTileCache.add(SunTileCache.java:257)
        at javax.media.jai.OpImage.addTileToCache(OpImage.java:1087)
        at javax.media.jai.OpImage.getTile(OpImage.java:1142)
        at javax.media.jai.PlanarImage.cobbleInt(PlanarImage.java:3153)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2178)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.calibration.gpf.calibrators.Sentinel1Calibrator.computeTile(Sentinel1Calibrator.java:533)
        at org.esa.s1tbx.calibration.gpf.CalibrationOp.computeTile(CalibrationOp.java:190)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.datamodel.Band.readRasterData(Band.java:311)
        at org.esa.snap.core.dataio.ProductSubsetBuilder.readBandRasterDataRegion(ProductSubsetBuilder.java:328)
        at org.esa.snap.core.dataio.ProductSubsetBuilder.readBandRasterDataImpl(ProductSubsetBuilder.java:297)
        at org.esa.snap.core.dataio.AbstractProductReader.readBandRasterData(AbstractProductReader.java:250)
        at org.esa.snap.core.gpf.common.SubsetOp.computeTile(SubsetOp.java:270)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)                         
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilters.Lee.performFiltering(Lee.java:116)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilters.Lee.computeTile(Lee.java:82)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilterOp.computeTile(SpeckleFilterOp.java:299)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.snap.core.gpf.common.BandMathsOp.fillSymbolWithData(BandMathsOp.java:355)
        at org.esa.snap.core.gpf.common.BandMathsOp.computeTile(BandMathsOp.java:322)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.getPixelValue(RangeDopplerGeocodingOp.java:1255)
        at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.computeTileStack(RangeDopplerGeocodingOp.java:1065)
        at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeRect(OperatorImageTileStack.java:116)
        at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeTile(OperatorImageTileStack.java:85)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.datamodel.Band.writeRasterDataFully(Band.java:384)
        at org.esa.snap.core.dataio.ProductIO.writeAllBands(ProductIO.java:448)
        at org.esa.snap.core.dataio.ProductIO.writeProduct(ProductIO.java:401)
        at org.esa.snap.core.dataio.ProductIO.writeProduct(ProductIO.java:330)
java.lang.NullPointerException
        at com.sun.media.jai.util.SunCachedTile.<init>(SunCachedTile.java:80)
        at com.sun.media.jai.util.SunTileCache.add(SunTileCache.java:257)
        at javax.media.jai.OpImage.addTileToCache(OpImage.java:1087)                                                      
        at javax.media.jai.OpImage.getTile(OpImage.java:1142)
        at javax.media.jai.PlanarImage.cobbleInt(PlanarImage.java:3153)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2178)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.calibration.gpf.calibrators.Sentinel1Calibrator.computeTile(Sentinel1Calibrator.java:533)
        at org.esa.s1tbx.calibration.gpf.CalibrationOp.computeTile(CalibrationOp.java:190)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.datamodel.Band.readRasterData(Band.java:311)
        at org.esa.snap.core.dataio.ProductSubsetBuilder.readBandRasterDataRegion(ProductSubsetBuilder.java:328)
        at org.esa.snap.core.dataio.ProductSubsetBuilder.readBandRasterDataImpl(ProductSubsetBuilder.java:297)
        at org.esa.snap.core.dataio.AbstractProductReader.readBandRasterData(AbstractProductReader.java:250)
        at org.esa.snap.core.gpf.common.SubsetOp.computeTile(SubsetOp.java:270)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilters.Lee.performFiltering(Lee.java:116)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilters.Lee.computeTile(Lee.java:82)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilterOp.computeTile(SpeckleFilterOp.java:299)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.snap.core.gpf.common.BandMathsOp.fillSymbolWithData(BandMathsOp.java:355)
        at org.esa.snap.core.gpf.common.BandMathsOp.computeTile(BandMathsOp.java:322)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.getPixelValue(RangeDopplerGeocodingOp.java:1255)
        at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.computeTileStack(RangeDopplerGeocodingOp.java:1065)
        at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeRect(OperatorImageTileStack.java:116)
        at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeTile(OperatorImageTileStack.java:85)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.datamodel.Band.writeRasterDataFully(Band.java:384)
        at org.esa.snap.core.dataio.ProductIO.writeAllBands(ProductIO.java:448)
        at org.esa.snap.core.dataio.ProductIO.writeProduct(ProductIO.java:401)
        at org.esa.snap.core.dataio.ProductIO.writeProduct(ProductIO.java:330)

Apologies for the huge paste job, but I was wondering whats causing this? It seems to work quite quickly with the caching so if I use the same WKT to subset on and use the same image, the process is done in seconds.

Just seems strange that the errors only appear after a few calls. Any help is appreciated!

Thanks,

Ciaran


Snappy doesn't clear memory cache
Temporary fix for snappy memory issues
#2

Further on to this, it seems to be an issue with caching after multiple calls.

The process seems to hang but then does eventually write out the .tiff, yet the error still occurs.

Seems caching only works a few times? Then I effectively have to process the whole image again without it…


#3

Bumping this with some more investigating.

I bumped my VM up to 8GB and also bumped up the java_max_mem property in snappy.ini to 8G

I’m processing some S1 imagery and subsetting it to POLYGON ((-4.136685132980347 50.36611236042063, -4.12860631942749 50.36611236042063, -4.12860631942749 50.37036895441764, -4.136685132980347 50.37036895441764, -4.136685132980347 50.36611236042063)) so its very small in terms of a S1 image.

Running the process once I get python hogging 16% of the VMs memory, making several calls it always idled at more memory:

16%, 28%, 42%, 52%, 81%, 81%, 95%, 95%, 90%, crashed.

I’m assuming this is to do with the Java side because memory just isn’t being freed. I’m removing all files in /tmp every time I run the process too so. Is there anyway to stop this continuous buildup of memory till eventual crashing?

Many thanks,

Ciaran


#4

Further to this, I’ve noticed posts mentioning forcing garbage collection via something such as jpy.get_type('java.lang.System').gc() and this doesn’t seem to do anything. It actually seems to increase the memory usage…


#5

Wondered if anyone has any suggestions RE this? Renders my pipeline useless after a few calls.


#6

@marpet sorry to pester! Do you know of any fixes/workarounds to this?


#7

Snappy is known to have memory issues, especially with clearing the memory after running a script. What might be worth a try is running your python script from Command Window, instead of Spyder (i particularly experienced problems with Spyder).


#8

The script is being running through a Flask application http://flask.pocoo.org so it’s not a issue with Spyder.

I’m hosting this as an api to call so having it effectively bomb out after a few calls isn’t handy at all, so far my only option is to write a cron job to kill the process and restart it, but that’s basically making the api useless as it could cut out when it’s running a job.

Frustrating that trying to force garbage collection doesn’t seem to help.


#9

No, I have no clue. Sorry. I also don’t see meaningful improvements coming very soon.
I still think that the major cause is the S1 data, respectively the S1 reader. But I’m not sure.


#10

That’s unfortunate to hear, I know it’s affecting a few users.

@seb is also experiencing issues with this. It makes creating apps hard/unmanageable as they eventually die from running out of memory.

Wonder if there’s anyone else who has suggestions for this?


#11

Hi Ciaran,

Garbage collection did not help me either. Maybe something related to this? I had a similar memory issues because I was invoking jpy.get_type() inside a loop. It is better to move out of the loop. It solved the issue for me.
Cheers
jose


#12

Thanks for the suggestion!

I managed to solve my issues with:

I’ll give yours a go sometime to see if it saves me doing my workaround!