Downloading data from SentinelSat

just wanted to go sure :wink:

Any idea what could have caused this error?

not really, sorry.

Hi,

I think I may have seen this error before. What happens if you make the second date one day later?

As in:
date = ('20190625', '20190626')

David

@dtlloyd it raises this error now:

---------------------------------------------------------------------------
SentinelAPIError                          Traceback (most recent call last)
<ipython-input-8-8263296e4025> in <module>()
     12                      platformname = 'Sentinel-2',
     13                      processinglevel = 'Level-2A',
---> 14                      cloudcoverpercentage = (0,5)
     15                      )

2 frames
/usr/local/lib/python3.6/dist-packages/sentinelsat/sentinel.py in query(self, area, date, raw, area_relation, order_by, limit, offset, **keywords)
    206         )
    207         formatted_order_by = _format_order_by(order_by)
--> 208         response, count = self._load_query(query, formatted_order_by, limit, offset)
    209         self.logger.info("Found %s products", count)
    210         return _parse_opensearch_response(response)

/usr/local/lib/python3.6/dist-packages/sentinelsat/sentinel.py in _load_query(self, query, order_by, limit, offset)
    347 
    348     def _load_query(self, query, order_by=None, limit=None, offset=0):
--> 349         products, count = self._load_subquery(query, order_by, limit, offset)
    350 
    351         # repeat query until all results have been loaded

/usr/local/lib/python3.6/dist-packages/sentinelsat/sentinel.py in _load_subquery(self, query, order_by, limit, offset)
    397                 # returned as the total results value when the query string was incorrect.
    398                 raise SentinelAPIError(
--> 399                     "Invalid query string. Check the parameters and format.", response
    400                 )
    401             total_results = int(json_feed["opensearch:totalResults"])

SentinelAPIError: HTTP status 200 OK: Invalid query string. Check the parameters and format.

Hi,

This code runs fine for me. I had to specify my own .json file. I have uploaded an alternate .json file here: Malta_Sicily.json (689 Bytes)

Perhaps you could try that .json file instead, to see if there is an issue with your .geojson file.

For reference, I am using version 0.13 of SentinelSat.

David

1 Like

@David I used your json file and this is what I got:

---------------------------------------------------------------------------
StopIteration                             Traceback (most recent call last)
<ipython-input-19-1da24c759dfe> in <module>()
      1 # download all results from the search
----> 2 api.download_all(products)

/usr/local/lib/python3.6/dist-packages/sentinelsat/sentinel.py in download_all(self, products, directory_path, max_attempts, checksum, n_concurrent_dl, lta_retry_delay)
    805 
    806         if len(failed_prods) == len(product_ids):
--> 807             raise next(iter(x.exception() for x in dl_tasks if x.exception()))
    808 
    809         return downloaded_prods, retrieval_scheduled, failed_prods

StopIteration: 

Could it be a bug in the sentinelsat 0.14? should I downgrade it?

I think that may be because there are no products covering that footprint on those dates. If you change the end date to ‘20190630’ there should be 11 available products to download (according to my copy of your code). Let me know if that is enough to start the download.

@dtlloyd No luck, same error. :frowning:

OK. I thought that would have fixed it. There are some other things you could try.

If you print
len(products)
do you get a number larger than 0?

If so, that suggests that
api.query()
is working fine. In which case, there could be something amiss with the products. You could try to check if one of the products you are requesting is online. Something like:

list_of_items = list(products.items()) # items from ordered dictionary product_ID = list_of_items[0][0] # item corresponding to product id product_info = api.get_product_odata(product_ID) # info for given product ID print(product_info['Online']) # True/false check for online

would check the first product in the dictionary called “products” and print “True” if that product is online.

1 Like

@dtlloyd len(products) returns 0

That’s surprising. For me:

footprint = geojson_to_wkt(read_geojson('Malta_Sicily.json')) 
products = api.query(footprint,
                         date = ('20190625','20190630'),
                         platformname = 'Sentinel-2',
                         processinglevel = 'Level-2A',
                         cloudcoverpercentage = (0, 10))
print(len(products))

returns a value of 11.

If you are running the exact same code and getting 0 I am not sure what is wrong, aside from the difference in SentinelSat versions. Sorry.

@dtlloyd where does it store the output? I mean when you run api.download_all(products)

I would imagine that is saves the files to the working directory, probably in the same folder where your Python script is. I prefer to specify a path to download the products to, since the files can be quite large. You can do this simply in the download_all function with:
api.download_all(products,directory_path=download_path)
where download_path is a string of the path to the folder you wish to download the products to.

Ok, thanks for the help. It works now I believe the error was instead of using 'Malta_Sicily.json' I used './'Malta_Sicily.json'.
One more question, is the number of images =len(products)? because in my case len(products)=5 and I got 5 L2A and 6 L2B despite that I specified processinglevel = 'Level-2A'

Good question. I believe that the output of api.query() is a dictionary with one entry for each product that satisfies the specifications provided to the query function. Hence the length of the dictionary should be the same as the number of products. However, there is probably a more sensible way to find out the number of products. I have only used SentinelSat for level 1 and level 2 Sentinel 3 products and it’s worked very well for that so far.

Thanks for the clarifications ! this was really helpful.

1 Like

I tired to download sentinel L2A data and I got the following error.
This is the code I used:

# connect to the API

api = SentinelAPI('Rym', 'thedevilwearsparada', 'https://scihub.copernicus.eu/dhus')

footprint = geojson_to_wkt(read_geojson('Bizerte_FieldCampagn_Location.geojson')) 

products = api.query(footprint,

                        date = ('20180401','20180705'),

                        platformname = 'Sentinel-2',

                        processinglevel = 'Level-2A',

                        cloudcoverpercentage = (0, 5))

print(len(products))

# download all results from the search
api.download_all(products)
---------------------------------------------------------------------------
CancelledError                            Traceback (most recent call last)
<ipython-input-21-1da24c759dfe> in <module>()
     1 # download all results from the search
----> 2 api.download_all(products)

1 frames
/usr/lib/python3.6/concurrent/futures/_base.py in exception(self, timeout)
   454         with self._condition:
   455             if self._state in [CANCELLED, CANCELLED_AND_NOTIFIED]:
--> 456                 raise CancelledError()
   457             elif self._state == FINISHED:
   458                 return self._exception

CancelledError:

The thing is that it went well when I downloaded the data for 2020 and 2019.

Can it be related to the fact that older images are stored in a different archive and can no longer be directly provided? ESA Copernicus data access - Long Term Archive and its drawbacks

Some discussions on this here:

Stumbled on this thread while trying to find out why setting same date as start and end date in sentinelsat api query gives no products in result. Although there are products on the date.
Fixed this issue by setting end date as yyyy-MM-ddThh:mm:ssZ (https://sentinelsat.readthedocs.io/en/latest/api_reference.html). Hope this helps someone.

from datetime import datetime

today = datetime.today().strftime(’%Y%m%d’)
print(today)
now = datetime.now().strftime("%Y-%m-%dT%H:%M:%SZ")
print(now)
footprint = geojson_to_wkt(read_geojson(‘path/to/your/footprint.geojson’))
products = api.query(footprint,
date=(today, now),
producttype=‘GRD’,
sensoroperationalmode=‘IW’)
print(products)

3 Likes