Band extraction error

Hello All, I tried to extract a single band (specifically, B3B2) using Raster → Bands extractor. The extraction claimed to be successful - .csv file has been written and new product appeared. However, the CSV file contains some garbage like:

978490 0.9481939
978491 0.9137262
883006 0.82021815
883007 0.8324185
978492 0.9346971
978493 0.8923883
883008 0.82240695978494 0.8756214

978495 0.91560537883009 0.7956432

978496 0.91859853
978497 0.9161481
978498 0.9216684
978499 0.8885484

i.e., “feature ID” is not ascending number and some rows are empty. Most of the file is correct though.
Second, attempting to create an image of the extracted band SNAP runs forever. Canceling the process gives the following diagnostics:
java.util.concurrent.CancellationException: Process terminated by user.
at org.esa.snap.core.datamodel.StxFactory.accumulate(StxFactory.java:325)
at org.esa.snap.core.datamodel.StxFactory.accumulate(StxFactory.java:297)
at org.esa.snap.core.datamodel.StxFactory.create(StxFactory.java:228)
at org.esa.snap.core.datamodel.StxFactory.create(StxFactory.java:275)
at org.esa.snap.core.datamodel.RasterDataNode.computeStxImpl(RasterDataNode.java:2468)
at org.esa.snap.core.datamodel.Band.computeStxImpl(Band.java:493)
at org.esa.snap.core.datamodel.RasterDataNode.getStx(RasterDataNode.java:2435)
at org.esa.snap.core.datamodel.RasterDataNode.createDefaultImageInfo(RasterDataNode.java:1839)
at org.esa.snap.core.datamodel.Band.createDefaultImageInfo(Band.java:472)
at org.esa.snap.core.datamodel.RasterDataNode.getImageInfo(RasterDataNode.java:1821)
at org.esa.snap.core.datamodel.RasterDataNode.getImageInfo(RasterDataNode.java:1804)
at org.esa.snap.core.image.ImageManager.prepareImageInfos(ImageManager.java:834)
at org.esa.snap.core.image.ColoredBandImageMultiLevelSource.create(ColoredBandImageMultiLevelSource.java:51)
at org.esa.snap.core.image.ColoredBandImageMultiLevelSource.create(ColoredBandImageMultiLevelSource.java:47)
at org.esa.snap.core.image.ColoredBandImageMultiLevelSource.create(ColoredBandImageMultiLevelSource.java:41)
at org.esa.snap.ui.product.ProductSceneImage.(ProductSceneImage.java:72)
at org.esa.snap.rcp.actions.window.OpenImageViewAction.createProductSceneImage(OpenImageViewAction.java:274)
at org.esa.snap.rcp.actions.window.OpenImageViewAction.access$100(OpenImageViewAction.java:67)
at org.esa.snap.rcp.actions.window.OpenImageViewAction$1.doInBackground(OpenImageViewAction.java:241)
at org.esa.snap.rcp.actions.window.OpenImageViewAction$1.doInBackground(OpenImageViewAction.java:220)
at com.bc.ceres.swing.progress.ProgressMonitorSwingWorker.doInBackground(ProgressMonitorSwingWorker.java:56)
at javax.swing.SwingWorker$1.call(SwingWorker.java:295)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at javax.swing.SwingWorker.run(SwingWorker.java:334)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused: java.util.concurrent.ExecutionException
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at javax.swing.SwingWorker.get(SwingWorker.java:602)
[catch] at org.esa.snap.rcp.actions.window.OpenImageViewAction$1.done(OpenImageViewAction.java:228)
at javax.swing.SwingWorker$5.run(SwingWorker.java:737)
at javax.swing.SwingWorker$DoSubmitAccumulativeRunnable.run(SwingWorker.java:832)
at sun.swing.AccumulativeRunnable.run(AccumulativeRunnable.java:112)
at javax.swing.SwingWorker$DoSubmitAccumulativeRunnable.actionPerformed(SwingWorker.java:842)
at javax.swing.Timer.fireActionPerformed(Timer.java:313)
at javax.swing.Timer$DoPostEvent.run(Timer.java:245)
at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311)
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:758)
at java.awt.EventQueue.access$500(EventQueue.java:97)
at java.awt.EventQueue$3.run(EventQueue.java:709)
at java.awt.EventQueue$3.run(EventQueue.java:703)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:728)
at org.netbeans.core.TimableEventQueue.dispatchEvent(TimableEventQueue.java:159)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)

Am I doing anything wrong or these are known bugs?
Thank you,
Yuri

Have you checked some of the topics here?

Exporting a whole band in CSV does not make much sense. You can select a few pixels and use Pixel Export, or use a mask of a ROI or variable (Export Mask Pixels). Why do you need you B2 in CSV?

1 Like

As I mentioned, I need to export B3B2 band - not B2 - as it is directly related to bathymetry, I need to export several thousand pixels, so specifying them manually is not possible. (If there is a way to import locations at which pixel values will be exported, I’d be grateful for explanation.) Thus, the only option I see is to export all pixels and then programmatically find the ones I need. However, saved CSV file contains garbage (I use S2TBX version 8.0.4 on Windows 10). I tried to save ALL bands, and the saved huge file also contains garbage.
The thread you referred to has no mentioning of incorrectly saved CSV files - possibly the bug was introduced in latest versions of SNAP.

What do you mean by B3B2? Is it a virtual band you have created? Is it the ratio of B3 and B2?

That the geo-coordinates are not exported is because the product has no raster holding those values.
They can be created by the Band Maths. Type LAT or LON as expression and set latitude or longitude as band name.
That the CSV output is not correct is noted. [SNAP-1487] Exporting product as CSV file creates corrupted output - JIRA (atlassian.net)

Another way how you can extract values to CSV is by creating a mask. Either draw the geometry on the image or import a shapefile or specify the WKT (Right click on image → “Geometry from WKT”).
Then select this mask and right click on image and select “Export Mask Pixels”. You can actually define the mask also by expression or value range.
You can store the data either to the clipboard or to a file (file is preferred if the number of pixels gets bigger).

Thank you - the reply sounds promising. Yes, the B3B2 band is virtual and is a ratio of logs of corresponding bands. Unfortunately, I do not understand how to use in my case Band Maths, or how to create a mask, or how to prepare well-known-text. Could you please refer to specific parts of the manual, related tutorials, or just explain in detail what should I do (any approach would do). I’m a novice with SNAP.
Thank you,
Yuri

Forgot to ask: although the spatial subset that I have is 2657x1385, the properties show that the raster is 2657x4155=2657x1385x3. The saved image as TIFF also has height of 4155. Where these 3 come from?

I prepared the mask using Mask manager, f(x), with the expression:
LAT > 18.289 && LAT < 18.414 && LON > -65.0650 && LON < -64.8135
The mask almost covers the subset, but seems ok. Clicking on the image opens the menu, but “Export Mask Pixels” is inactive. Should I save a mask somehow to make it a separate band?

Sometimes the action is not properly activated. Try to select the geometry in the Product Tree. You find it under “Vector Data”.

Wher do you see this information?

Under “Vector Data” there are only “pins”, “ground_control_points” and “geometry”. The latter opens an empty window.
Correction about “Export Mask Pixels”: it is inactive only in “Extracted bands”. In full product it is active and did save all points in a mask. Took very long, as the action saved all bands, while I needed only latitude, longitude and B3B2 band. It doesn’t seem to give a chance to choose values to save.
Maybe there are other ways to save what I need? (I will reprocess the saved file, of course, but it’s awkward.)

You should create a subset (selected bands) before doing the mask export. Then you only get what is in the product.

I did create a subset (only B3B2 band) using “Extract bands”. As I mentioned above, in this product (it is virtual and has “[5]” in front of the name which means that it hasn’t been saved on disk, as I understand) when I click on the image, “Export Mask Pixels” is not active. Do I have to save it and reload to make this item active? Will the product have raster that holds geo-coordinates?
Actually, when I save it and open as a new product, SNAP says that it is already opened, and do I want to open it as a separate product. I do want, but it appears with “[*]” in front, as other virtual products.

You can convert your virtual band to a non-virtual first using “Convert band” by clicking the right button of the the mouse over the name of the band.

Unfortunately, the option “Convert band” is greyed out, so I cannot click on it.

Sorry to raise the question again, but when I’m repeating all the processing now, including creation of a mask that covers all the subset, “File → Export → Other → Mask pixels” is greyed out, so the pixel values under the mask cannot be saved. Obviously, last time I did some operation that made it possible without noticing it. “Vector data” under the Product tree does not have “geometry”, only “pins” and “ground_control_points”, both empty. Could you please help? I am lost.
All I need is just save all pixels of one band (or all bands) in a readable/parseable form.

Sorry to come back to this but, did you try to use the subset function to save spatial and band subsetting at the same time?

I’m not sure what you mean. Choosing “Raster → Subset”? The band I “subsetted” is not virtual, but “Mask pixels” option remains unavailable.
I wonder if anybody ever tried to save band values together with their lat-lon’s in a readable form?

Let’s start over. You can do these two tings:

  1. resample the S2 bans to the same resolution–>calculate B3B2 with BandMaths–> draw a geometry ->export pixels within the geometry in CSV. Lat/lon are saved here (see pic)

2 . resample the S2 bans to the same resolution–>calculate B3B2 with BandMaths–>use the Raster/Subset function and select the spatial subset by lat/lon and the bands you wat to export in dimap–>from the new image save as CSV. You will only have lat lon in the CSV if you have created the bands as @marpet explained above.

1 Like

This is exactly what I am doing. The mask is created using Mask manager and an expression defining limits for lat/lon. “File → Export → Other → Mask pixels” is still greyed out. The only detail that I might be missing is “select the mask”. How should I do this?

Put the cursor over the geometry (it will change colours). Once is selected, you should be able to export all pixels in the mask.

I’m sorry, but what does “geometry” mean? The mask itself? The cursor should be a “hand” or anything else? Currently with hand, nothing changes color, and all I can do is to move the image. In Mask manager the current mask is selected (tickmark on the left).