Snappy not freeing memory


#1

Hi there,

I’ve developed a python api for generating flood mapping with snappy. When I call it a few times I occasionally get a large stack trace as follows:

    java.lang.NullPointerException                                                                                           
        at com.sun.media.jai.util.SunCachedTile.<init>(SunCachedTile.java:80)
        at com.sun.media.jai.util.SunTileCache.add(SunTileCache.java:257)
        at javax.media.jai.OpImage.addTileToCache(OpImage.java:1087)
        at javax.media.jai.OpImage.getTile(OpImage.java:1142)
        at javax.media.jai.PlanarImage.cobbleInt(PlanarImage.java:3153)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2178)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.calibration.gpf.calibrators.Sentinel1Calibrator.computeTile(Sentinel1Calibrator.java:533)
        at org.esa.s1tbx.calibration.gpf.CalibrationOp.computeTile(CalibrationOp.java:190)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.datamodel.Band.readRasterData(Band.java:311)
        at org.esa.snap.core.dataio.ProductSubsetBuilder.readBandRasterDataRegion(ProductSubsetBuilder.java:328)
        at org.esa.snap.core.dataio.ProductSubsetBuilder.readBandRasterDataImpl(ProductSubsetBuilder.java:297)
        at org.esa.snap.core.dataio.AbstractProductReader.readBandRasterData(AbstractProductReader.java:250)
        at org.esa.snap.core.gpf.common.SubsetOp.computeTile(SubsetOp.java:270)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilters.Lee.performFiltering(Lee.java:116)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilters.Lee.computeTile(Lee.java:82)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilterOp.computeTile(SpeckleFilterOp.java:299)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)                                                            
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.snap.core.gpf.common.BandMathsOp.fillSymbolWithData(BandMathsOp.java:355)
        at org.esa.snap.core.gpf.common.BandMathsOp.computeTile(BandMathsOp.java:322)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.getPixelValue(RangeDopplerGeocodingOp.java:1255)
        at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.computeTileStack(RangeDopplerGeocodingOp.java:1065)
        at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeRect(OperatorImageTileStack.java:116)
        at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeTile(OperatorImageTileStack.java:85)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.datamodel.Band.writeRasterDataFully(Band.java:384)
        at org.esa.snap.core.dataio.ProductIO.writeAllBands(ProductIO.java:448)
        at org.esa.snap.core.dataio.ProductIO.writeProduct(ProductIO.java:401)
        at org.esa.snap.core.dataio.ProductIO.writeProduct(ProductIO.java:330)
java.lang.NullPointerException
        at com.sun.media.jai.util.SunCachedTile.<init>(SunCachedTile.java:80)
        at com.sun.media.jai.util.SunTileCache.add(SunTileCache.java:257)
        at javax.media.jai.OpImage.addTileToCache(OpImage.java:1087)
        at javax.media.jai.OpImage.getTile(OpImage.java:1142)
        at javax.media.jai.PlanarImage.cobbleInt(PlanarImage.java:3153)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2178)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.calibration.gpf.calibrators.Sentinel1Calibrator.computeTile(Sentinel1Calibrator.java:533)       
        at org.esa.s1tbx.calibration.gpf.CalibrationOp.computeTile(CalibrationOp.java:190)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.datamodel.Band.readRasterData(Band.java:311)
        at org.esa.snap.core.dataio.ProductSubsetBuilder.readBandRasterDataRegion(ProductSubsetBuilder.java:328)
        at org.esa.snap.core.dataio.ProductSubsetBuilder.readBandRasterDataImpl(ProductSubsetBuilder.java:297)
        at org.esa.snap.core.dataio.AbstractProductReader.readBandRasterData(AbstractProductReader.java:250)
        at org.esa.snap.core.gpf.common.SubsetOp.computeTile(SubsetOp.java:270)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilters.Lee.performFiltering(Lee.java:116)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilters.Lee.computeTile(Lee.java:82)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilterOp.computeTile(SpeckleFilterOp.java:299)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.snap.core.gpf.common.BandMathsOp.fillSymbolWithData(BandMathsOp.java:355)
        at org.esa.snap.core.gpf.common.BandMathsOp.computeTile(BandMathsOp.java:322)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)                                                            
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.getPixelValue(RangeDopplerGeocodingOp.java:1255)
        at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.computeTileStack(RangeDopplerGeocodingOp.java:1065)
        at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeRect(OperatorImageTileStack.java:116)
        at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeTile(OperatorImageTileStack.java:85)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.datamodel.Band.writeRasterDataFully(Band.java:384)
        at org.esa.snap.core.dataio.ProductIO.writeAllBands(ProductIO.java:448)
        at org.esa.snap.core.dataio.ProductIO.writeProduct(ProductIO.java:401)
        at org.esa.snap.core.dataio.ProductIO.writeProduct(ProductIO.java:330)
java.lang.NullPointerException
        at com.sun.media.jai.util.SunCachedTile.<init>(SunCachedTile.java:80)
        at com.sun.media.jai.util.SunTileCache.add(SunTileCache.java:257)
        at javax.media.jai.OpImage.addTileToCache(OpImage.java:1087)
        at javax.media.jai.OpImage.getTile(OpImage.java:1142)
        at javax.media.jai.PlanarImage.cobbleInt(PlanarImage.java:3153)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2178)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.calibration.gpf.calibrators.Sentinel1Calibrator.computeTile(Sentinel1Calibrator.java:533)
        at org.esa.s1tbx.calibration.gpf.CalibrationOp.computeTile(CalibrationOp.java:190)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.datamodel.Band.readRasterData(Band.java:311)
        at org.esa.snap.core.dataio.ProductSubsetBuilder.readBandRasterDataRegion(ProductSubsetBuilder.java:328)
        at org.esa.snap.core.dataio.ProductSubsetBuilder.readBandRasterDataImpl(ProductSubsetBuilder.java:297)           
        at org.esa.snap.core.dataio.AbstractProductReader.readBandRasterData(AbstractProductReader.java:250)
        at org.esa.snap.core.gpf.common.SubsetOp.computeTile(SubsetOp.java:270)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilters.Lee.performFiltering(Lee.java:116)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilters.Lee.computeTile(Lee.java:82)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilterOp.computeTile(SpeckleFilterOp.java:299)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.snap.core.gpf.common.BandMathsOp.fillSymbolWithData(BandMathsOp.java:355)
        at org.esa.snap.core.gpf.common.BandMathsOp.computeTile(BandMathsOp.java:322)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.getPixelValue(RangeDopplerGeocodingOp.java:1255)
        at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.computeTileStack(RangeDopplerGeocodingOp.java:1065)
        at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeRect(OperatorImageTileStack.java:116)
        at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeTile(OperatorImageTileStack.java:85)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)                               
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.datamodel.Band.writeRasterDataFully(Band.java:384)
        at org.esa.snap.core.dataio.ProductIO.writeAllBands(ProductIO.java:448)
        at org.esa.snap.core.dataio.ProductIO.writeProduct(ProductIO.java:401)
        at org.esa.snap.core.dataio.ProductIO.writeProduct(ProductIO.java:330)
java.lang.NullPointerException
        at com.sun.media.jai.util.SunCachedTile.<init>(SunCachedTile.java:80)
        at com.sun.media.jai.util.SunTileCache.add(SunTileCache.java:257)
        at javax.media.jai.OpImage.addTileToCache(OpImage.java:1087)
        at javax.media.jai.OpImage.getTile(OpImage.java:1142)
        at javax.media.jai.PlanarImage.cobbleInt(PlanarImage.java:3153)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2178)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.calibration.gpf.calibrators.Sentinel1Calibrator.computeTile(Sentinel1Calibrator.java:533)
        at org.esa.s1tbx.calibration.gpf.CalibrationOp.computeTile(CalibrationOp.java:190)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.datamodel.Band.readRasterData(Band.java:311)
        at org.esa.snap.core.dataio.ProductSubsetBuilder.readBandRasterDataRegion(ProductSubsetBuilder.java:328)
        at org.esa.snap.core.dataio.ProductSubsetBuilder.readBandRasterDataImpl(ProductSubsetBuilder.java:297)
        at org.esa.snap.core.dataio.AbstractProductReader.readBandRasterData(AbstractProductReader.java:250)
        at org.esa.snap.core.gpf.common.SubsetOp.computeTile(SubsetOp.java:270)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)                         
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilters.Lee.performFiltering(Lee.java:116)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilters.Lee.computeTile(Lee.java:82)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilterOp.computeTile(SpeckleFilterOp.java:299)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.snap.core.gpf.common.BandMathsOp.fillSymbolWithData(BandMathsOp.java:355)
        at org.esa.snap.core.gpf.common.BandMathsOp.computeTile(BandMathsOp.java:322)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.getPixelValue(RangeDopplerGeocodingOp.java:1255)
        at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.computeTileStack(RangeDopplerGeocodingOp.java:1065)
        at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeRect(OperatorImageTileStack.java:116)
        at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeTile(OperatorImageTileStack.java:85)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.datamodel.Band.writeRasterDataFully(Band.java:384)
        at org.esa.snap.core.dataio.ProductIO.writeAllBands(ProductIO.java:448)
        at org.esa.snap.core.dataio.ProductIO.writeProduct(ProductIO.java:401)
        at org.esa.snap.core.dataio.ProductIO.writeProduct(ProductIO.java:330)
java.lang.NullPointerException
        at com.sun.media.jai.util.SunCachedTile.<init>(SunCachedTile.java:80)
        at com.sun.media.jai.util.SunTileCache.add(SunTileCache.java:257)
        at javax.media.jai.OpImage.addTileToCache(OpImage.java:1087)                                                      
        at javax.media.jai.OpImage.getTile(OpImage.java:1142)
        at javax.media.jai.PlanarImage.cobbleInt(PlanarImage.java:3153)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2178)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.calibration.gpf.calibrators.Sentinel1Calibrator.computeTile(Sentinel1Calibrator.java:533)
        at org.esa.s1tbx.calibration.gpf.CalibrationOp.computeTile(CalibrationOp.java:190)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.datamodel.Band.readRasterData(Band.java:311)
        at org.esa.snap.core.dataio.ProductSubsetBuilder.readBandRasterDataRegion(ProductSubsetBuilder.java:328)
        at org.esa.snap.core.dataio.ProductSubsetBuilder.readBandRasterDataImpl(ProductSubsetBuilder.java:297)
        at org.esa.snap.core.dataio.AbstractProductReader.readBandRasterData(AbstractProductReader.java:250)
        at org.esa.snap.core.gpf.common.SubsetOp.computeTile(SubsetOp.java:270)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilters.Lee.performFiltering(Lee.java:116)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilters.Lee.computeTile(Lee.java:82)
        at org.esa.s1tbx.sar.gpf.filtering.SpeckleFilterOp.computeTile(SpeckleFilterOp.java:299)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.snap.core.gpf.common.BandMathsOp.fillSymbolWithData(BandMathsOp.java:355)
        at org.esa.snap.core.gpf.common.BandMathsOp.computeTile(BandMathsOp.java:322)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
        at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.cobbleFloat(PlanarImage.java:3254)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2181)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
        at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
        at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.getPixelValue(RangeDopplerGeocodingOp.java:1255)
        at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.computeTileStack(RangeDopplerGeocodingOp.java:1065)
        at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeRect(OperatorImageTileStack.java:116)
        at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeTile(OperatorImageTileStack.java:85)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
        at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.datamodel.Band.writeRasterDataFully(Band.java:384)
        at org.esa.snap.core.dataio.ProductIO.writeAllBands(ProductIO.java:448)
        at org.esa.snap.core.dataio.ProductIO.writeProduct(ProductIO.java:401)
        at org.esa.snap.core.dataio.ProductIO.writeProduct(ProductIO.java:330)

Apologies for the huge paste job, but I was wondering whats causing this? It seems to work quite quickly with the caching so if I use the same WKT to subset on and use the same image, the process is done in seconds.

Just seems strange that the errors only appear after a few calls. Any help is appreciated!

Thanks,

Ciaran


Snappy doesn't clear memory cache
Temporary fix for snappy memory issues
#2

Further on to this, it seems to be an issue with caching after multiple calls.

The process seems to hang but then does eventually write out the .tiff, yet the error still occurs.

Seems caching only works a few times? Then I effectively have to process the whole image again without it…


#3

Bumping this with some more investigating.

I bumped my VM up to 8GB and also bumped up the java_max_mem property in snappy.ini to 8G

I’m processing some S1 imagery and subsetting it to POLYGON ((-4.136685132980347 50.36611236042063, -4.12860631942749 50.36611236042063, -4.12860631942749 50.37036895441764, -4.136685132980347 50.37036895441764, -4.136685132980347 50.36611236042063)) so its very small in terms of a S1 image.

Running the process once I get python hogging 16% of the VMs memory, making several calls it always idled at more memory:

16%, 28%, 42%, 52%, 81%, 81%, 95%, 95%, 90%, crashed.

I’m assuming this is to do with the Java side because memory just isn’t being freed. I’m removing all files in /tmp every time I run the process too so. Is there anyway to stop this continuous buildup of memory till eventual crashing?

Many thanks,

Ciaran


#4

Further to this, I’ve noticed posts mentioning forcing garbage collection via something such as jpy.get_type('java.lang.System').gc() and this doesn’t seem to do anything. It actually seems to increase the memory usage…


#5

Wondered if anyone has any suggestions RE this? Renders my pipeline useless after a few calls.


#6

@marpet sorry to pester! Do you know of any fixes/workarounds to this?


#7

Snappy is known to have memory issues, especially with clearing the memory after running a script. What might be worth a try is running your python script from Command Window, instead of Spyder (i particularly experienced problems with Spyder).


#8

The script is being running through a Flask application http://flask.pocoo.org so it’s not a issue with Spyder.

I’m hosting this as an api to call so having it effectively bomb out after a few calls isn’t handy at all, so far my only option is to write a cron job to kill the process and restart it, but that’s basically making the api useless as it could cut out when it’s running a job.

Frustrating that trying to force garbage collection doesn’t seem to help.


#9

No, I have no clue. Sorry. I also don’t see meaningful improvements coming very soon.
I still think that the major cause is the S1 data, respectively the S1 reader. But I’m not sure.


#10

That’s unfortunate to hear, I know it’s affecting a few users.

@seb is also experiencing issues with this. It makes creating apps hard/unmanageable as they eventually die from running out of memory.

Wonder if there’s anyone else who has suggestions for this?


#11

Hi Ciaran,

Garbage collection did not help me either. Maybe something related to this? I had a similar memory issues because I was invoking jpy.get_type() inside a loop. It is better to move out of the loop. It solved the issue for me.
Cheers
jose


#12

Thanks for the suggestion!

I managed to solve my issues with:

I’ll give yours a go sometime to see if it saves me doing my workaround!


#13

at least, it closes the file and enables windows preview


#14

Hi,

Thank you very much Ciaran and Marco.

I have the same problem. The only effective solution that worked in my case was the one of @Ciaran_Evans (launching and independent process).

@marpet is there any open ticket or similar about the issue?


#15

Same issue here, especially when using the S2sampler


#16

No there is no special ticket for memory issue in snappy, but there are some others
https://senbox.atlassian.net/browse/SNAP-869
https://senbox.atlassian.net/browse/SITBX-562
https://senbox.atlassian.net/browse/SITBX-482

I’ve created one now. (SNAP-960). For me it is not yet clear where the memory issues comes from. Maybe it is within SNAP, maybe in jpy or in snappy. This needs to be investigated.
However, I can tell that the further development of SNAP and the Toolboxes will start again soon. And then we will tackle such memory and performance issues. Finally… :slight_smile:
But till the fixes will be released it still might take a few month.


Snappy doesn't clear memory cache
Snappy GRD geocoding is 5 times slower than gpt graph
#17

@lumaro - glad it fixed your issue (for now)

Thanks @marpet for raising the issue! Hopefully the workaround can be useful for people at the moment, I know here at work our Remote Sensing folk have hit the issue too.

@harmel I’d recommend giving the workaround ago, it’s by no means pretty but it allows for bulk processing. Give me a shout if you need a hand!


#18

Thanks a lot @Ciaran_Evans .
I noticed that image data are loaded by chunk of 512 pixel rows. So for me the work-around is to run the process for each 512-row chunk independently.
Cheers,
Tristan