Changes to Copernicus Sentinels data access - Copernicus Data Space Ecosystem

Please take note of planned changes to the Copernicus Sentinels data access. This will affect how everyone accesses Copernicus Sentinel data, both inside and outside SNAP.

If you have any related questions ESA experts intend to answer them in this thread.

5 Likes

On a related note, Copernicus DEM 30m and 60m resolution are now available via direct download without need of logging in:

1 Like

The change from the current distribution to the new one (Copernicus Data Space) is certainly a major step if one considers the 2023 roadmap. This will be a long journey and ESA would like to embark the community on that trip.

However, this will also represent a disruption on the way to query/access/download data. Most of the scripts you have written (including mines) or the packages like sentinelsat will have to be adjusted.

Please have a try with the new APIs and let us know about your experience, issues, recommendation, etc.

Kind Regards,
Nuno Miranda
Sentinel-1 Mission Manager

3 Likes

Some preliminary feedback on using this new API and the web browser.

The Sentinel-1 Collection seems to be accessible in the API and in the browse (but not viewable in the browser)

It seems there are Landsat-5 products accessible from the API. They are not visible in the web browser (neither queryable, nor viewable)

The complete list of collections does not seem to be documented. Is there a web service to query for the list of available collections ?

The query interface in the browser does not allow to filter by dates including hours.

The description of potential query parameters supported for S1 with OpenSearch is available here:
https://catalogue.dataspace.copernicus.eu/resto/api/collections/Sentinel1/describe.xml
From this list of parameters:

  • Querying using publishedAfter is extremely slow. This much slower than querying using startDate.
  • Filtering by platform (S1A vs S1B) does not seem to work properly
  • There is no way to filter on processor name and/or processor version
  • The processing baseline is not documented. I am not sure it is possible to document it as simple number for S1.

That’s only first feedbacks. There may be some other points to notice.

1 Like

I can not export sentinal 1 images to my google derive. My code used to work before!! It is an error called Internall Error(Error 13).
I use google earth code engine.
Dataset:“‘COPERNICUS/S1_GRD’”

This forum is about the SNAP software by ESA, not Sentinel data in general or its access by external services.

According to this example nothing has changed with the access to S1 GRD collection. Maybe it was a connection error.

The official pages recommend to report the error with the given link when you encounter an Internal Error.

2 Likes

Really nice features! Congrats.
The platform seems intuitive. I’ve explored the timelapse and compare panel (which is very intersting) and I would suggest (given only my 15 min experience there) to add a tick box in the compare panel to each product that has been added. For instance if you have only two products, thats not important, but if we want to compare for example 4 products (like natural colors, moisture, SWIR and NDSI) it would benefit to rapidly activate or deactivate (normally we want only two at the time) the stack of products we don’t want to compare at any moment.

I would all so suggest to add for each layer of the products panel visualization (and more even to the compare panel) to add a color bar scale with values range (also if possible, some kind mouse pointer activation over any pixel of the intrinsic value for the stack products being compared).

My “2 cents” for now! :saluting_face:

2 Likes

At present, has the method of accessing information changed yet? and still able to download via sentinelsat or not? , I’m very confused.

This forum is about the SNAP software by ESA, not Sentinel data in general or its access by external services.

Could you recommend where the best place to discuss accessing Copernicus Sentinel data would be more appropriate?

At present converting my process from sentinelsat is at a complete standstill so seeing how the community is changing their workflow would be very helpful.

Cheers, thanks,
-Andrea

3 Likes

Hi
Welcome to our group!!!

1 Like

@mengdahl , @ABraun

Is there any update for which SNAP 9 or SNAP 10 will have the new dependencies related to S1 orbits etc?

Since a couple of days ago the scihub repo for S1 orbit download does not work anymore.

Best,

1 Like

No update should be necessary as we’ve handled this issue at the back-end as we have our own orbits-repository. When SNAP9 looks for orbits from scihub and cannot find them, it should fall-back to our orbits collection.

Hello,

S1tbx deburst seems to not work since the ecosystem. I am not sure if this thread is the right place but i feel like its more related to the ecosystem than it is to S1TBX. Help is much appreciated:)
I tried 3 different S1 SLC files downloaded with a python script or with the new browser. All give me a java.lang.NullPointerException when i try running the operator with snappy in python or with the snap GUI.
It works fine on images that i downloaded from the old platform.
Processing: Calibrate, Deburst.
Images:
S1A_IW_SLC__1SDV_20231103T173914_20231103T173941_051056_06280A_F9F4.SAFE
S1A_IW_SLC__1SDV_20220111T055257_20220111T055324_041409_04EC72_B7CA.SAFE
S1B_IW_SLC__1SDV_20210520T171203_20210520T171230_026991_03397F_8945.SAFE

I am running a fresh snap 9.0.8 and S1TBX 9.0.3, no updates available, same problem on two different PCs.

The old images do not have SAFE at the end… What does that mean?

The Error:

java.lang.NullPointerException
	at org.esa.snap.engine_utilities.datamodel.AbstractMetadata.getBandAbsMetadata(AbstractMetadata.java:384)
	at org.esa.s1tbx.commons.Sentinel1Utils.getCalibrationVectorList(Sentinel1Utils.java:934)
	at org.esa.s1tbx.commons.Sentinel1Utils.getCalibrationPixel(Sentinel1Utils.java:960)
	at org.esa.s1tbx.sentinel1.gpf.TOPSARDeburstOp.getMergedPixels(TOPSARDeburstOp.java:680)
	at org.esa.s1tbx.sentinel1.gpf.TOPSARDeburstOp.updateCalibrationVector(TOPSARDeburstOp.java:640)
	at org.esa.s1tbx.sentinel1.gpf.TOPSARDeburstOp.updateOriginalMetadata(TOPSARDeburstOp.java:580)
	at org.esa.s1tbx.sentinel1.gpf.TOPSARDeburstOp.updateTargetProductMetadata(TOPSARDeburstOp.java:415)
	at org.esa.s1tbx.sentinel1.gpf.TOPSARDeburstOp.initialize(TOPSARDeburstOp.java:142)
Caused: org.esa.snap.core.gpf.OperatorException
	at org.esa.snap.engine_utilities.gpf.OperatorUtils.catchOperatorException(OperatorUtils.java:434)
	at org.esa.s1tbx.sentinel1.gpf.TOPSARDeburstOp.initialize(TOPSARDeburstOp.java:145)
	at org.esa.snap.core.gpf.internal.OperatorContext.initializeOperator(OperatorContext.java:528)
	at org.esa.snap.core.gpf.internal.OperatorContext.getTargetProduct(OperatorContext.java:298)
	at org.esa.snap.core.gpf.Operator.getTargetProduct(Operator.java:385)
	at org.esa.snap.core.gpf.GPF.createProductNS(GPF.java:333)
	at org.esa.snap.core.gpf.GPF.createProduct(GPF.java:308)
	at org.esa.snap.core.gpf.GPF.createProduct(GPF.java:287)
	at org.esa.snap.graphbuilder.rcp.dialogs.SingleOperatorDialog.createTargetProduct(SingleOperatorDialog.java:175)
[catch] at org.esa.snap.graphbuilder.rcp.dialogs.SingleOperatorDialog.onApply(SingleOperatorDialog.java:287)
	at org.esa.snap.ui.AbstractDialog.lambda$initUI$6(AbstractDialog.java:519)
	at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2022)
	at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2348)
	at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:402)
	at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259)
	at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:252)
	at java.awt.AWTEventMulticaster.mouseReleased(AWTEventMulticaster.java:289)
	at java.awt.Component.processMouseEvent(Component.java:6539)
	at javax.swing.JComponent.processMouseEvent(JComponent.java:3324)
	at java.awt.Component.processEvent(Component.java:6304)
	at java.awt.Container.processEvent(Container.java:2239)
	at java.awt.Component.dispatchEventImpl(Component.java:4889)
	at java.awt.Container.dispatchEventImpl(Container.java:2297)
	at java.awt.Component.dispatchEvent(Component.java:4711)
	at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4904)
	at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4535)
	at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4476)
	at java.awt.Container.dispatchEventImpl(Container.java:2283)
	at java.awt.Window.dispatchEventImpl(Window.java:2746)
	at java.awt.Component.dispatchEvent(Component.java:4711)
	at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:760)
	at java.awt.EventQueue.access$500(EventQueue.java:97)
	at java.awt.EventQueue$3.run(EventQueue.java:709)
	at java.awt.EventQueue$3.run(EventQueue.java:703)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:84)
	at java.awt.EventQueue$4.run(EventQueue.java:733)
	at java.awt.EventQueue$4.run(EventQueue.java:731)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
	at java.awt.EventQueue.dispatchEvent(EventQueue.java:730)
	at org.netbeans.core.TimableEventQueue.dispatchEvent(TimableEventQueue.java:159)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
	at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
	at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
1 Like

@jun_lu , @djagula can you have a look at this?
Thanks

A JIRA ticket ([SNAP-3578] - JIRA) has been created to track the issue. We will look into it. Thank you for reporting the problem.

1 Like

Unfortunately, the orbit file download (auto download) fails in SNAP9…

java.io.IOException: No valid orbit file found for 01-OCT-2019 16:49:54.382951
Orbit files may be downloaded from The Sentinels Scientific Data Hub
and placed in C:\Users\12345.snap\auxdata\Orbits\Sentinel-1\POEORB\S1A\2019\10

1 Like

We are currently investigating the orbit files download issue. We’ll keep you updated on any developments.

1 Like

Really lovely features! Congratulations.
The interface looks user-friendly. After spending only fifteen minutes exploring the fascinating timelapse and comparison panel, I recommend adding a checkbox to the board for each newly added product. For example, if you only have two products, that doesn’t matter. However, if we want to compare, say, four products (natural colours, moisture, SWIR, and NDSI), it would be beneficial to quickly activate or deactivate the stack of products we don’t want to compare at that particular time (we usually only want two at a time).
I recommend adding a colour bar scale with a values range to each layer of the product panel visualisation (and even more to the compare panel). If feasible, you should additionally activate the mouse cursor over any pixel representing the intrinsic value of the stack items being compared. For more details infycletechnologies

Dear @Gerald,
Sentinel-1 Toolbox version 9.0.4 is now available. Please check for updates using the HelpCheck for Updates menu.

1 Like