Like others watching this thread, we have customers who are relying on SNAP-enhanced Sentinel-1 imagery for their operations. Could you give a rough estimate of when the update will be available? We’d like to keep our customers informed as to when they can expect enhanced images again.
Many thanks!
Paul
We should have the update out in 1-2 days.
Thanks for your reply! We’ll pass the information on.
The update is out, please test it!
Hi nuno.miranda
Tested new SNAP version, and got nearly the same result as with Matlab previously, but I trust SNAP more here. I guess I did not do all interpolations as SNAP does.
Your result on scaling of EW1 noise data is interesting (SNAP does not seem to do this scaling). This gives images that are good for visual and automatic classifications, but is absolute level of sigma0 still correct? Of course you dont need abs sigma0 in all applications, its important when comparing S-1 sigma0 data to theoretical backscattering models, or to data from other sensors, like RS-2.
Cheers,
Marko
Thermal noise removal and GRD-Border-Noise are now working after the update. Thank you.
Note this change for slice assembled products: “Error: Noise removal should be applied prior to slice assembly”. This precludes use of Product-Reader for multiple source images when using Thermal noise removal. This complicates graphs with 2 or more input images in an orbit, requiring separate graphs using Read(s) for one, two, three, etc. images.
Can you please upload the update into the maven repository (http://nexus.senbox.net/nexus/content/repositories/public/org/esa/)? If I am not wrong, the version is still the old one. Thanks
Many thanks for the update! We’ve been able to process EW Sentinel1 images successfully (thermal noise removal, calibration, speckle filtering and terrain correction). Unfortunately, IW images are giving us NullPointerExceptions. Just using thermal noise removal results in this traceback:
RuntimeError: java.lang.NullPointerException
Oddly enough, when we use thermal noise removal and calibration, we get a more detailed traceback:
java.lang.NullPointerException
at com.sun.media.jai.util.SunCachedTile.<init>(SunCachedTile.java:80)
at com.sun.media.jai.util.SunTileCache.add(SunTileCache.java:257)
at javax.media.jai.OpImage.addTileToCache(OpImage.java:1087)
at javax.media.jai.OpImage.getTile(OpImage.java:1142)
at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
at org.esa.s1tbx.calibration.gpf.calibrators.Sentinel1Calibrator.computeTile(Sentinel1Calibrator.java:533)
at org.esa.s1tbx.calibration.gpf.CalibrationOp.computeTile(CalibrationOp.java:190)
at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
at org.esa.snap.core.datamodel.Band.writeRasterDataFully(Band.java:384)
at org.esa.snap.core.dataio.ProductIO.writeAllBands(ProductIO.java:448)
at org.esa.snap.core.dataio.ProductIO.writeProduct(ProductIO.java:401)
at org.esa.snap.core.dataio.ProductIO.writeProduct(ProductIO.java:306)
We get the above error from this SAFE archive: S1B_IW_GRDH_1SDH_20180325T024234_20180325T024257_010182_0127FD_9686.SAFE.zip and we’re using SNAP via the Python bindings (just in case that helps). The behaviour is the same when processing either the HH or HV images within the SAFE archive.
Does anyone have any ideas as to what could ge going wrong here? Should I open a new issue in GitHub for this problem?
Many thanks for your help,
Paul
The update seems to work for us.
Dear mengdahl and others,
Thank you for your SNAP update. Processing of IW images works fine, except when using SliceAssembly. In that case I get java heap errors with S1 images processed with IPF v 2.90. Older S1 images do work fine with SliceAssembly.
An other effect I noticed is that background values in Terrain Corrected Gamma0 output are negative numbers instead of zeroes. Most background pixels have value -0.00123456784058 (sic!), but also other negative power values occur. I am testing now if these negative values also occur without radiometric calibration.
Cheers,
Rob
Thanks Paul, I’ll look into it.
Here’s some more information, just in case it can help narrow down the debugging search:
If you need any further details from me, just let me know.
Regards,
Paul
With the new version we get the following errors when using thermal noise removal:
Java heap spacejava.lang.NullPointerException
java.lang.NullPointerException
java.lang.NullPointerException
When running from the command line I got a little more information:
....10%....20%....30%....40%...GC overhead limit exceeded
java.lang.NullPointerException
90% done.
org.esa.snap.core.gpf.OperatorException: GC overhead limit exceeded
at org.esa.snap.core.gpf.graph.GraphProcessor$GPFImagingListener.errorOccurred(GraphProcessor.java:363)
at com.sun.media.jai.util.SunTileScheduler.sendExceptionToListener(SunTileScheduler.java:1646)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:921)
at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:461)
at org.esa.snap.raster.gpf.LinearTodBOp.computeTile(LinearTodBOp.java:116)
at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:80)
at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:407)
at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:393)
at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:73)
at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at com.sun.media.jai.util.RequestJob.compute(SunTileScheduler.java:247)
at com.sun.media.jai.util.WorkerThread.run(SunTileScheduler.java:468)
Caused by: org.esa.snap.core.gpf.OperatorException: GC overhead limit exceeded
at org.esa.snap.engine_utilities.gpf.OperatorUtils.catchOperatorException(OperatorUtils.java:432)
at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.computeTileStack(RangeDopplerGeocodingOp.java:1091)
at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeRect(OperatorImageTileStack.java:116)
at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeTile(OperatorImageTileStack.java:85)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
... 21 more
Error: GC overhead limit exceeded
GC overhead limit exceeded
GC overhead limit exceeded
java.lang.NullPointerException
This was for scene: S1A_IW_GRDH_1SDV_20180328T054153_20180328T054218_021211_024795_51D5.zip
We are running a graph consisting of subset->thermal noise removal->apply-orbit-file->terrain_correction->lin2db->band-select->write. Without thermal noise removal everything runs through.
I noticed unusual memory usage applying thermal noise removal on some Sentinel products (for example S1B_IW_GRDH_1SDV_20180401T183211_20180401T183240_010293_012B95_D592.zip and a few more images downloaded after 25th march).
Could you please investigate possible memory overflow issues in this operator?
Thank you very much in advance.
Hi,
i made the same observation: A simple Slicea assemby with thermal noise removal does not work anymore with more than 5 tiles.
I receive a java heap space error. I have 64 GB of RAM. It worked well with 5 tiles and even before March 13th also with 6 tiles or more.
With the change of preprocessing the demand for memory is much stronger. Could you please make the routine memory friendly? Not everybody has server access to more than 64 GB Ram!
Thank you
Holger
I also have troubles getting my processing chain working because of memory issues.
Even 100GB on a cluster machine is not enough for the SliceAssembly-step of 4 or more tiles. (Noise Removal also requires much more memory than before March). Changing the settings -Xms -Xmx to sensible values in the gpt.vmoptions does not help either.
I’d appreciate if this issue would be addressed soon.
I’m also having issues with GPT at the moment. I have a polarimetry workflow that works in the SNAP GUI but not in GPT.
GPT gives no errors but hangs so I assume it’s a memory issue? Maybe there is a better way to debug GPT (I will try with -e).
My machine has 64GB RAM and I have allocated up to 40GB.
Tryin to run Sentinel-1 ESD corregistration. Using graph builder on GUI works fine but when trying the same graph on GPT I have the java.lang.NullPointerException. I am working on Linux 16.04 with two S1 SLC IW images.
M
Similar failure to noise remove on IPF 2.9 IW GRD scenes.
SNAP v6.0.2 does not produce valid sigma0 after noise removal.
Granule (of after March 13, 2018) S1A_IW_GRDH_1SDV_20180314T152304_20180314T152333_021013_02414B_59E7.zip downloaded from the Hub (and tested on many other granules globally)
[1] OS: CentOS 6.8 linux
SNAP version 6.0.2 GUI, updated now
Procedure: Radar radiometric cal --> S1 thermal noise removal.
Calibration works well, producing sigma0. But the output of noise removal has blank sigma0 all over the scene for both VH and VV (min(sigma0)=0, max(sigma0)=0).
[2]OS: MacOS 10.12.6
SNAP version 6.0.2 GUI, updated now
“java.lang.NullPointerException” is what I got. The process fails to complete.
[3] SNAP V5 produces calibrated noise corrected sigma0 on IW GRD before Mar 2018
[4] Mixed success. SNAPv6.0.2 on CentOS linux 250GB ram. Started from SLC. Noise removal on IPF2.9 data (March 13, 2018) works. One scene however has many artifacts, unrelated to noise removal (granule names are provided).
S1A_IW_GRDH_1SDV_20180323T001321_20180323T001347_021135_02452C_426C_Cal_Deb_ML_Spk_SRGR
S1A_IW_GRDH_1SDV_20180314T152304_20180314T152334_021013_02414B_1C0A_Cal_Deb_ML_Spk_SRGR
@lveci could you check it?