Sentinel-2: weeding out bad products?

Amongst other examples, I found these two products published for the same Sentinel-2 Single-Tile acquired on the same date at the same time :

The latter looks to be the good one, but the former is present in the archive and I’d like to automate its suppression using my own scripts.

Q: Are there any flags in the Sentinel-2 Product metadata to help me do this?

My current idea is to parse the published time and favour the latest - i.e. 20190305T175734 over the other (20190305T162851)

Hi,
some L1C products are cut in two halves, with a little overlap, which provides you two products on the same date. In this case, the overlap is sufficient to allow you discarding the first one, but it is not always the same, for instance if the cut is in the middle of the image.

Initially, ESA was saying they would deliver “consolidated” L1C products that would merge both parts, but it was 4 years ago, and it is not done yet, so maybe they have given up.

Olivier

Thanks Olivier, but the above are specifically Level-2A products.

But that’s the same: 1L1C => 1 L2A

Your right of course. The L2A simply reflects the L1C product occurrence.
But this is an ongoing situation witness this example from March 2019


S2A_MSIL1C_20190305T101021_N0207_R022_T32SMJ_20190305T153109


S2A_MSIL1C_20190305T101021_N0207_R022_T32SMJ_20190305T171354

where it seems to me that the former product was aborted and meant to be replaced by the latter - and as such contains no additional useful information,
and IMHO would better to be withdrawn from the archive.

As I said, for some reasons ESA has to cut its L1C products. They do it properly with an overlap, and provide each chunk of the same tile with the same acquisition date.

For instance, on the same date, here is the tile to the east of the one you displayed. It is cut too, and you don’t want to discard any of these as they don’t provide fully redundant information.

https://peps.cnes.fr/rocket/#/search?collection=S2ST&processingLevel=LEVEL2A&latitudeBand=S&utmZone=32&mgrsGSquare=NJ&maxRecords=50&startDate=2019-03-05T00:05:00&completionDate=2019-03-06T11:05:00&page=1

Thanks OHagolle

I know:
In fact four of the tiles I’m interested on that date:
32SMJ 32SNJ 32TMK 32TNK
all have similar ‘bad’ duplicates - and considering these tiles as a set I can’t see I am loosing any pixels by retaining only those published later rather than earlier.
But I’m ready to be convinced otherwise.

Hey, you are hard to convince :wink:

No, it’s not fully duplicates.
if you only keep this one,
https://peps.cnes.fr/rocket/#/collections/S2ST/ec8f6862-7ae2-58b9-b1e5-5f59e677fa31

you will lose the Northeast corner, which is contained in this one
https://peps.cnes.fr/rocket/#/collections/S2ST/a8ee55f4-0ad1-5980-a852-507fac992034

And by the way, did you notice that Theia provides high quality L2A products over the whole Italy, including Sardigna, since 2016, and in near real time.
https://theia.cnes.fr/atdistrib/rocket/#/search?page=1&q=cagliari&collection=SENTINEL2

Wow that is good news.

Update: Okay - I am convinced by your example.
Why! Oh why! do ESA have so many corner cases that need to be programmed around!
The UTM tile idea turns out to be such a pain when you have mosaic all these pixels from the the same swath back together again.

BTW: Theia don’t stock either product from the example date 20190305 I used above!
https://theia.cnes.fr/atdistrib/rocket/#/search?page=1&collection=SENTINEL2&platform=SENTINEL2A&startDate=2019-02-01&completionDate=2019-04-04&location=T32SMJ&relativeOrbitNumber=022

Yes ! :wink:

Well ESA and Copernicus already do a lot, but still leave us little things to complain about… The Frenchies like that :wink:

And the tiles have a few cons, but a lot of pros, they enable to process time series much more easily

Theia only delivers products with less than 90% of clouds, to save space, download time and computing resource: it reduces the amount of products by 1/3.
Olivier

Relatedly, but a different special case:

For 2018.10.03 there are two almost identical products available for relative orbit 022 (at least over Sardinia)
They seem to differ in something called processing baseline - of which I know little but usually increments gradually over time -

• the former has a N0209 baseline
S2B_MSIL2A_20181031T101139_N0209_R022_
and one split tile 32TNL

• while the latter has a N0211 baseline
S2B_MSIL2A_20181031T101139_N0211_R022_
and no split tiles.

Q: Is it okay simply to favour the latter over the former?

(Ps: I’ve only ever encountered one case of such a ‘repeat product’ )

Hello @gbrelstaff

This News Item might help you.

You can fins out more about the evolution of the processing baseline from Section 3.1 of the L2A Data Quality Report that we produce every month, and is made available from the S2 User Guide Document Library.

Cheers

Jan

S2 MPC Operations Manager

Thanks @Jan

I guess that the S2B_MSIL2A_20181031T101139_N0211_R022_ product was an operational test that went well!
I notice that the N0210 baseline that followed that test lasted only a month - before incrementing properly to N0211 - was there some fix to be made?

Perhaps you might also explain a bit about the split tiles I’ve encountered (even during the N0211 baseline). Is there a technical reason why they haven’t been patched back to form a single product before presenting them to us? Or why I shouldn’t do so for a local archive?

Hello,

The version of Processing Baseline (PROBAS) is incremented each time there is a significant update - evolution - of an instrument processing facility (IPF), or an update of a set of auxiliary data files (ADFs), or both.

Thus, 02.11 is a later evolution of the PROBAS, and therefore is an improvement on 02.09. Again, the specificity of these evolutions to the processing baseline are documented in the Data Quality Report (DQR).

The PROBAS is a GIPP - a Ground Image Processing Parameters file. To quote the Product Specification Document (PSD), these files are a set of XML files that are associated to each
processing component so as to define a particular set of parameters and their values.

The reprocessing of Tile 32TNL with 02.11 can also be identified in the increment of the date in the full filename:

S2B_MSIL2A_20181031T101139_N0209_R022_T32TNL_20181107T144242

Download URL: https://scihub.copernicus.eu/dhus/odata/v1/Products(‘83007dd6-7af9-40de-8d4a-15a50ee6b527’)/$value

S2B_MSIL2A_20181031T101139_N0211_R022_T32TNL_20190116T215303

Download URL: https://scihub.copernicus.eu/dhus/odata/v1/Products(‘9e831fa6-95bc-4b99-88cd-963df50c4e79’)/$value

The ‘split’ Tiles to which you refer are presumably split becuse they sit across the finish and start of two datastrips within a datatake, in this manner:

Cheers

Jan

Just to share the knowledge:

The following shell script code uses ImageMagick’s convert to sew back together the pairs of jpg quicklooks for split tiles (while showing the join):

      convert quicklook1.jpg    -fuzz 2%  -transparent black  aaa.png
      convert quicklook2.jpg    -fuzz 2%  -transparent black  bbb.png
      convert aaa.png bbb.png -composite  result.jpg

Furthermore, you can use those intermediate png files also to blend the images on a web pages as follows:

<div style="position:relative;text-align:right;"><img src="aaa.png" width="512" height="512" border="1"><img src="bbb.png" style="position:absolute;top:-512px;left:0px;"width="512" height="512" border="1"></div>