There is a number of difference for which a NRT-3h is different than a FAST-24h.
I understand the image posted by kristov is sigma0 but it is not clear if it dB or natural values (difference in dB, etc). Whatever information on how the difference or ratio is made would help.
I can cite many different reasons why this is happening starting from the orbit which is what triggers this thread:
The state vector being different, the sat speed is different, the FM rate is different, focussing is different, the pixels are different
The state vector being different, the terrain correction is different, the pixels are different
etc.
The state vector are different the geocoding is different, you are not comapring exactly the same pixels hence there are differences
In order to further investigate we need the filename of the products to have a look to the annotations.
The difference mentioned above were in power units, not in decibels. In the snapshots below, differences are in dB (i.e. abs(db1 - db2)). Color scale is from 0-5dB, although in this subset, differences up to more than 20 dB were observed. It’s clear that the difference is not huge everywhere, but it’s there, sometimes it’s big, and it seems to be warping-related, so probably indeed the state vectors which have a small impact on geocoding the signal, so the difference images highlights built-up areas with large backscatter gradients.
For your information, these are the products that were used to produce the Sigma0 backscatter using S1Toolbox:
So yes, we understand we’re basically comparing apples to oranges. But this is exactly why we’re worried by this policy change of disseminating only one product (without proper notice to users). As is clear from your explanation and our comparisons, it’s basically different products, but how are we supposed to provide consistent products and derived time series to our end users when there’s a sudden (more or less silent) change in timeliness category with basically a different end product as a result? This is pretty bad when you’re doing time series analyses and stuff.
I am not sure that the a difference in orbits (that differs from few cms) could lead to 20dB error even punctual. This is counter intuitive and goes against our understanding (at least mine).
Would it be possible for you to make available the NRT-3h product? I think, this one is not available anymore in the PDGS once the FAST-24h is available.
In order to rule out error in post-processing s/w (SNAP), we will look at the data using our own tools in native geometry (no geocoding performed).
Do you confirm, that the geocoding is done with the same orbit (e.g. RESORB or POEORB) or you are using the orbit annotated in the product?
Hi @nuno.miranda,
I’m not sure where we got both product versions. At least one comes from Creodias. Processing to Sigma0 should have happened with exactly the same processing graph, where we specifically request restituted orbits. But I need to check internally with the colleagues who did the processing of the two files so they can confirm. Until that time, we need to assume that maybe some settings could explain the differences, as they are beyond what you expect. Let me get back to you asap.
So, it seems that “Delays in Sentinel-1 processing” in case of Fast-24h is an indefinite one. Let’s see how these NRT-3h products work with the predicted orbits as they are today (in the early days of Sentinel-1 there seemed to be some geometry issues). The delay in informing users about the product changes was one month.
A related question for @nuno.miranda and other ESA folks.
Does this change me that one doesn’t need to explicitly work with PREORB files for the occasional case where RESORB files spanning certain SLC products are not available and that one can just work with state vectors within the annotation since they essentially came from PREOB/ RESORB (hopefully unmodified from source) products to begin with.
The S-1 products are annotated with the orbit used for processing (in the XML file).
We made a change (several years back in the past) to annotate the exact same state vectors as the input source. This should not be the case since many years now. In addition, we provide at least state vectors prior / after the acutal sensing to allow for a proper interpolation. Previously, the processor was interpolating to 1s the input source (sampled at 10s) leading to some noise.
The PREORB was gradually introduced last year. It is a specific prediction made few hours before sensing covering few hours. Thus, the prediction is very accurate and from the tests made comparing with RESORB and POEORB we were in the same bullpark <10cm. This means that for applications based on GRD products (20m spacing) it should not make a big difference from radiometric and geometric point of view. For people using SLC for INSAR purposes, using the best orbit source (RESORB or POEORB) is (I believe) the preferred choice as it ensures a more stable performance.
The news published makes reference to another point which could relate to the issue of Kristofvt.
Previously, NRT-3h were processed in “non-slicing” mode (this is annotated in the manifest) while FAST-24h is processed in slicing mode.
In slicing mode, the full data-take (e.g. 10 min long) is available. We use this information to process each individual slice (25s chunks) with more information allowing to use internal calibration efficiently and to sample the bursts of ALL slices in the same data-take the same way.
In non-silcing mode, the seamless sampling is not guaranteed, the internal calibration is not used to best extent and the noise annotation are less accurate. This could lead to few changes in the radiometry (internal cal) but not to the extent of kristovft. This is why I am curious. This explains why NRT-3h were reprocessed in FAST-24h to achieve seamless quality across the slices.
Now I understand, that since recently the processing NRT-3h has improved and the data could be processed in slicing all the time (annotated in the manifest). The unique difference being the RESORB / PREORB.
EOSupport was mentioned in this thread. For the sake of completeness, I got the information linked to by kristofvt above from EOSupport by e-mail just before kristofvt posted the link to this thread.
@nuno.miranda we reprocessed the older Fast-24h product to make sure GPT settings were exactly the same, but the result is identical, including the differences we observed.
According to my colleague, both the Fast24h and NRT3h products should still be accessible on CreoDIAS:
During the weekend, there seems to be yet another turn in the processing policy of GRDH products. Most images are as two copies, both with timeliness NRT-3h. Very little difference in the manifest.SAFE file, just the second product computed about an hour later than the first one. I just wonder why is this ?
Is it intentional to litter the scihub with two copies of the same image if the differences are next to null and the user has no option (except the processing time) to differentiate between them ?
The duplicate problem seems to have been there only during that weekend. After that, I have not noticed duplicates, and the number of scenes per day seems to be about the same as before 24.2.2021.
I finally managed to get a working creodias account last week, Actually, I am using one of theior account as the registration procedure was giving issues.
I checked the products on my side.
From the annotation of each product it appears that:
The NRT-3 product is processed using the PREORB (as expected) in a processing mode called NRT Slicing mode, not taken into account the complete acquisition context.
The Fast-24 product is processed using the RESORB (as expected) in Slicing mode.
On average there is no bias in radiometry between the two images.
However, it looks in deed that there are punctual very large differences near large gradient of NRCS.
See on this image: composition of blue (NRCS from the image on NRT-3 / NRT Slicing mode) and red ( Difference of NRCS between the two images)
As the products were not processed equally, they are not sampled in the same grid, which here has larger impact for NRCS on “non distributed” targets (then near NRCS gradients).
This does not mean that one of the two products is bad, but :
point wise that they are just sampled differently
this one processed in NRT will not be smoothly continuous with previous/next slice on the same datatake, at the contrary to the one processed in FAST-24
Products acquired before 23rd feb 2021 processed in NRT do not consider the complete data take during processing.
This is the case for this product.
This should not be the case after the 23rd of feb 2021.
For me this is only a sampling issue.
The point we discuss here is on GRD products for which the detection prevents to perform accurate interpolation between the sampling point.
For fine point target measurements (including PS Cal) it must be done on SLC images with interpolation of the impulse response, thus not impacted to my knowledge.
