ESA Copernicus data access - Long Term Archive and its drawbacks

Microwave Toolbox
#11

This means a 1 image per hour image retrieval if I’m right?

Thanks a lot for your developments

#12

My latest info is that users can request a product from the long term archive every 30 minutes. But this is a quota set by the Copernicus open access hub.

For the sake of completeness, ESA also provides a download script that deals with products in the long term archive https://scihub.copernicus.eu/userguide/BatchScripting#dhusget_script.

4 Likes
Problems trying to download LTA Sentinel-1 images
#13

thanks for the link.

Quentin

#14

Did anyone find any stable solution? I have a list of product IDs now how I can download them using a script?

#15

I think the best you can do is to use alternative mirrors such as PEPS or ASF.

I contacted the eosupport for the Sentinel platform but their response doesn’t fit with my needs :

[…] you try an bypass the quota restrictions by using multiple user accounts to access large volumes of data products from the LTA. Please note that the maximum number of products that a single user can request on SciHub is 1 every 30 minutes. An additional quota limit is applied to users of the APIHub of maximum 20 products every 12 hours. These restrictions are dictated by ESA and unless they change their policy then users will have to follow these guidelines. […]

So I guess you will not find what you need using the “official” sci-hub.

Note : in their response, they pointed out the importance this type of requests in order to understand people needs. So feel free to contact them to let them know the issue.

3 Likes
#16

Hi, your branch looks looks really useful to workaround this issue. I use sentinelsat (not a dev) and am wondering if you plan to merge this back in via pull request if its working well for you? Cheers

#17

Upstreaming would be great and is also my goal. I still have to debug some issues though. Another problem is, that it currently relies on some Python3 only features.

1 Like
#18

So this will continue running and giving the error # 429 Too Many Requests until it has downloaded each imagery (one per half hours mark)?

#19

The LTA code is now part of sentinelsat https://github.com/sentinelsat/sentinelsat. But there hasn’t been a new release yet, so you have to install from the master branch.

Documentation on how to use it is below.

#20

SNAP8.0 provides access to ASF as well so for older products that should provide a workaround. It would be possible to improve the Copernicus Hub access but my impression has been that most users use standalone tools for access in any case(?).

1 Like
#21

@mengdahl Excellent news ! Will it be possible to use GPT to download images with SNAP8 ?

#22

Hi

Seems that the LTA policy affect also several tools, among them DIAS platforms. For me, as am using Sentinel 2 data for now, a workaround solution is the use of EarthExplorer of USGS which has all S2 images from 2015 onwards (at least for my region where i work now).

Is quite annoying that official, public paid tools are not working as expected …

Hope that this will change soon.

D.

1 Like
#23

I am also working with S1 time-series and try to somehow automate making “full use” of my user Quota by writing python scripts.
The links provided in this post seem to be very useful and I will check it out soon.
However I am just starting with using python so I face some issues here and there.

I would like to discuss these in this forum, that I really like so far for getting really quick and on point replies and finding a lot of resources browsing through old posts.
Is it therefore possible to open a sub forum e.g. on data retrieval for that?
It seems that this Info is relatively spread, but it would be thematically fitting here, because obviously you need data for working with SNAP or the modules from the Sentinel-toolboxes and manual downloading is not an option for many up to date scientfic tasks like time series extraction or environmental parameter retrieval. And it is like dpoursanidis and others pointed out that ESA should be interested in enabling users for access to all the sentinel data anytime.

Cheers and thanks for the good work everybody,

K.

#24

You can open a new topic in the #development section, although some already exist, e.g. Downloading data for snap using sentinelsat API - LTA error

In this forum we surely have little impact on the data dissemination strategies of ESA, but of course its a good platform for exchange how others deal with the drawbacks of the LTA.

2 Likes
Sentinel 2 long term archive
#25

In addition to what @ABraun says, ESA Copernicus support is quite easy to join (eosupport@copernicus.esa.int). Don’t hesitate to give a feedback, it’s part of their job.

Cheers,

Quentin

3 Likes
#26

Hello,

I am trying to download ~1200 images from Sentinel-1A from ESA open access hub, and put all the images in the cart. But when I try to download the cart, the website gives an error message. The image is attached below. I tried Google Chrome, Firefox, and Explorer but I get similar errors. Can you please tell me how to batch download?

Thank you!

Problem
Problem1272×133 20 KB

#27
#28

Thank you! How do we use it? I downloaded the asf_search using cmd, but I don’t understand the code. Can you help me?

#29

Please have a look at the official documentation: Basics - ASF SAR Data Search Manual
No need to code anything yourself, you just have to adjust the search variables.

#30

I read the page multiple times, but still don’t understand how to use the code. I edited the sample script provided in the overview as follows:

import asf_search as asf

results = asf.granule_search([‘ALPSRS279162400’, ‘ALPSRS279162200’])
print(results)

wkt = ‘POLYGON((-113.94 47.96, -113.94 44.34, -110.53 44.34, -110.53 47.96))’
results = asf.geo_search(platform=[asf.PLATFORM.SENTINEL1], intersectsWith=wkt, maxResults=2000)
print(results)

But I think the this is intended for ALOS PALSAR. How should I change it to Sentinel-1A? What should I do now? and how should I run the Python script? I know very little of Python.