Changes to Copernicus Sentinels data access - Copernicus Data Space Ecosystem

We are currently investigating the orbit files download issue. We’ll keep you updated on any developments.

1 Like

Really lovely features! Congratulations.
The interface looks user-friendly. After spending only fifteen minutes exploring the fascinating timelapse and comparison panel, I recommend adding a checkbox to the board for each newly added product. For example, if you only have two products, that doesn’t matter. However, if we want to compare, say, four products (natural colours, moisture, SWIR, and NDSI), it would be beneficial to quickly activate or deactivate the stack of products we don’t want to compare at that particular time (we usually only want two at a time).
I recommend adding a colour bar scale with a values range to each layer of the product panel visualisation (and even more to the compare panel). If feasible, you should additionally activate the mouse cursor over any pixel representing the intrinsic value of the stack items being compared. For more details infycletechnologies

Dear @Gerald,
Sentinel-1 Toolbox version 9.0.4 is now available. Please check for updates using the HelpCheck for Updates menu.

1 Like

Are the products that you downloaded from the new ecosystem in zip file format? If yes, could you load the product into SNAP and check how many swathes it has? In my case, if I load the compressed product directly into SNAP, the product has only two swathes (IW1 and IW2) and IW3 is missing. I think it could be the reason for the exception. However, if I decompress the product first and load it into SNAP by using the manifest.safe file, then the product has all three swathes and I can deburst it without problem. We will continue investigating the issue.

I read through the opensearch API code for orbit file search and download, currently it is hardcoded to point to the currently offline OData endpoint :

static final String COPERNICUS_ODATA_ROOT = scihub dot copernicus dot eu …;

This should be updated to something like:
static final String COPERNICUS_ODATA_ROOT = catalogue dot dataspace dot copernicus dot eu/odata/v1";

Will have to test the actual query against the OData service to ensure the correct orbit files are identified.

Cheers,

Tisham.

1 Like

Yes, they are in .zip format and yes IW3 is missing, as i described here S-1 Tops Split Error: Operation Exception. Index=size=0 - #4 by Unmanaged0292.

As i also described deleting the quicklook jpg image from the .zip file solved the problem with reading the IW3 product. It was the only difference to the zip files i downloaded from the old system. I did not ckeck all bytes from the tiff files though, but all the .xds and .xml are similar.
Unpacking the folder and reading the manifest.safe also worked.

This problem occurs with downloads from asf as well as from ecosystem.

Unfortunately the problem does not stop there. When i apply the calibration IW3 goes missing again. When i then apply deburst snap fails to create the image and has several processes running on 100% CPU for a while and says waiting thread received a null tile:

java.lang.RuntimeException: Waiting thread received a null tile.
	at com.sun.media.jai.util.SunTileScheduler.scheduleTile(Unknown Source)
	at javax.media.jai.OpImage.getTile(Unknown Source)
	at org.esa.snap.core.datamodel.StxFactory.accumulateTile(StxFactory.java:351)
	at org.esa.snap.core.datamodel.StxFactory.accumulate(StxFactory.java:335)
	at org.esa.snap.core.datamodel.StxFactory.accumulate(StxFactory.java:297)
	at org.esa.snap.core.datamodel.StxFactory.create(StxFactory.java:201)
	at org.esa.snap.core.datamodel.StxFactory.create(StxFactory.java:275)
	at org.esa.snap.core.datamodel.RasterDataNode.computeStxImpl(RasterDataNode.java:2468)
	at org.esa.snap.core.datamodel.Band.computeStxImpl(Band.java:493)
	at org.esa.snap.core.datamodel.RasterDataNode.getStx(RasterDataNode.java:2435)
	at org.esa.snap.core.datamodel.RasterDataNode.createDefaultImageInfo(RasterDataNode.java:1839)
	at org.esa.snap.core.datamodel.Band.createDefaultImageInfo(Band.java:472)
	at org.esa.snap.core.datamodel.RasterDataNode.getImageInfo(RasterDataNode.java:1821)
	at org.esa.snap.core.datamodel.RasterDataNode.getImageInfo(RasterDataNode.java:1804)
	at org.esa.snap.core.image.ImageManager.prepareImageInfos(ImageManager.java:834)
	at org.esa.snap.core.image.ColoredBandImageMultiLevelSource.create(ColoredBandImageMultiLevelSource.java:51)
	at org.esa.snap.core.image.ColoredBandImageMultiLevelSource.create(ColoredBandImageMultiLevelSource.java:47)
	at org.esa.snap.core.image.ColoredBandImageMultiLevelSource.create(ColoredBandImageMultiLevelSource.java:41)
	at org.esa.snap.ui.product.ProductSceneImage.<init>(ProductSceneImage.java:74)
	at org.esa.snap.rcp.actions.window.OpenImageViewAction.createProductSceneImage(OpenImageViewAction.java:274)
	at org.esa.snap.rcp.actions.window.OpenImageViewAction.access$100(OpenImageViewAction.java:67)
	at org.esa.snap.rcp.actions.window.OpenImageViewAction$1.doInBackground(OpenImageViewAction.java:241)
	at org.esa.snap.rcp.actions.window.OpenImageViewAction$1.doInBackground(OpenImageViewAction.java:220)
	at com.bc.ceres.swing.progress.ProgressMonitorSwingWorker.doInBackground(ProgressMonitorSwingWorker.java:56)
	at javax.swing.SwingWorker$1.call(SwingWorker.java:295)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at javax.swing.SwingWorker.run(SwingWorker.java:334)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused: java.util.concurrent.ExecutionException
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at javax.swing.SwingWorker.get(SwingWorker.java:602)
[catch] at org.esa.snap.rcp.actions.window.OpenImageViewAction$1.done(OpenImageViewAction.java:228)
	at javax.swing.SwingWorker$5.run(SwingWorker.java:737)
	at javax.swing.SwingWorker$DoSubmitAccumulativeRunnable.run(SwingWorker.java:832)
	at sun.swing.AccumulativeRunnable.run(AccumulativeRunnable.java:112)
	at javax.swing.SwingWorker$DoSubmitAccumulativeRunnable.actionPerformed(SwingWorker.java:842)
	at javax.swing.Timer.fireActionPerformed(Timer.java:313)
	at javax.swing.Timer$DoPostEvent.run(Timer.java:245)
	at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311)
	at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:758)
	at java.awt.EventQueue.access$500(EventQueue.java:97)
	at java.awt.EventQueue$3.run(EventQueue.java:709)
	at java.awt.EventQueue$3.run(EventQueue.java:703)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
	at java.awt.EventQueue.dispatchEvent(EventQueue.java:728)
	at org.netbeans.core.TimableEventQueue.dispatchEvent(TimableEventQueue.java:159)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
	at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
	at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)

When i download Data from S3 i get IW3 after calibration and can open IW3 but after deburst it fails with either a null tile or data buffer:

java.lang.RuntimeException: Cannot construct DataBuffer.
	at com.sun.media.jai.util.DataBufferUtils.constructDataBuffer(Unknown Source)
	at com.sun.media.jai.util.DataBufferUtils.createDataBufferFloat(Unknown Source)
	at javax.media.jai.ComponentSampleModelJAI.createDataBuffer(Unknown Source)
	at javax.media.jai.RasterFactory.createWritableRaster(Unknown Source)
	at javax.media.jai.PlanarImage.createWritableRaster(Unknown Source)
[catch] at javax.media.jai.SourcelessOpImage.computeTile(Unknown Source)
	at com.sun.media.jai.util.SunTileScheduler.scheduleTile(Unknown Source)
	at javax.media.jai.OpImage.getTile(Unknown Source)
	at javax.media.jai.PlanarImage.cobbleFloat(Unknown Source)
	at javax.media.jai.PlanarImage.getData(Unknown Source)
	at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
	at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:449)
	at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:435)
	at org.esa.snap.core.gpf.Operator.getSourceTile(Operator.java:459)
	at org.esa.s1tbx.sentinel1.gpf.TOPSARDeburstOp.computeTileInOneSwathFloat(TOPSARDeburstOp.java:904)
	at org.esa.s1tbx.sentinel1.gpf.TOPSARDeburstOp.computeTileStack(TOPSARDeburstOp.java:811)
Caused: org.esa.snap.core.gpf.OperatorException: Cannot construct DataBuffer.
	at org.esa.snap.engine_utilities.gpf.OperatorUtils.catchOperatorException(OperatorUtils.java:434)
	at org.esa.s1tbx.sentinel1.gpf.TOPSARDeburstOp.computeTileStack(TOPSARDeburstOp.java:826)
	at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeRect(OperatorImageTileStack.java:122)
[catch] at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeTile(OperatorImageTileStack.java:86)
	at com.sun.media.jai.util.SunTileScheduler.scheduleTile(Unknown Source)
	at javax.media.jai.OpImage.getTile(Unknown Source)
	at com.sun.media.jai.util.SunTileScheduler.compute(Unknown Source)
	at com.sun.media.jai.util.TileJob.compute(Unknown Source)
	at com.sun.media.jai.util.WorkerThread.run(Unknown Source)

Currently I have a process that runs automatically to download sentinel-2 images at L1C level for tiles, I need them in the .SAFE format so that they can enter the atmospheric correction process with sen2cor, however, I have not been able to access the images As I did with the sentinelsat library, the example scripts download via AOI. Is there any way to download the complete images with the new API?

1 Like

Noted! Thanks! That was a quick fix!
Just updating it now…

1 Like

Hello, SNAP stopped working for SAR data, it cannot apply the orbit file anymore.

Error: [NodeId: Apply-Orbit-File] No valid orbit file found for 14-JUL-2023 18:10:56.536773
Orbit files may be downloaded from …scihub.copernicus.eu/gnss/odata/v1/
and placed in /home/myuser/.snap/auxdata/Orbits/Sentinel-1/POEORB/S1A/2023/07

How can I make SNAP downloading the orbit files automatically again? Thanks

1 Like

Hello,

Just tested and it works now. Do you still have this problem?

1 Like

Yes, this is my SNAP. Should I reinstall it?

Hello,

To summarize the current workaround is then to unzip the data and load from the manifest.safe file, right?
I am asking to make sure I am not missing a SNAP update or something else.

Thanks!

1 Like

Yes, you are right.

No need to reinstall SNAP. You just need to install the latest updates : HelpCheck for Updates.
The current SNAP version is:
image

and S1TBX:
image

1 Like

Thanks a lot!

1 Like

Checking for updates fixed the orbit files download, thank you

1 Like

Hello,

I already update the SNAP to the current version, but when I try to apply orbit on sentinel-1 data in SNAP this error occurs:

WARNING: org.esa.s1tbx.orbits.gpf.ApplyOrbitFileOp: No valid orbit file found for 03-JAN-2024 11:32:46.972227
Orbit files may be downloaded from https://scihub.copernicus.eu/gnss/odata/v1/
and placed in C:\Users\00024988\.snap\auxdata\Orbits\Sentinel-1\POEORB\S1A\2024\01

image

this my current s1tbx version:
image

can you help this?

thanks

Does anyone know which is the copernicus spacedata user that needs to be entered in the SNAP library?.

THNAKS A LOT

Hello,
It is the same user that you use to log in to the Copernicus Browser .

Thanks Diana.

My user is Miguel Romero, but i try login but dosent work.