Dear forum members and all who joined the discussion.
Please excuse me for my English.
After long correspondences by mail and various discussions and thinking, I decided to summarize.
I thank all participants of this process who was able to help me.
I will be glad to continue the discussion on this issue.
So.
Here is one of my favorite responses from one of the leading experts telespazio.com, expert
Jérôme LOUIS (Earth Observation Missions Expert, Sentinel-2 Expert Support Laboratory L2A Leader):
“… Concerning your particular request, it can happen that the TOA reflectance values are higher than 1.0. It usually happens for bright, high altitude clouds or for some particular solar/viewing geometry of acquisition, e.g. specular reflection on inclined artificial roofs, it could also happen on slopes covered by snow facing the sun and the Sentinel-2 sensor. …”
Yes this is true. This explains the fact that the values of the TOA Reflectance can be greater than 1.
In support of this, you can refer to the few publications on this issue.
For example:
Yale Guide to Landsat 8 Image Processing
http://www.yale.edu/ceo/Documentation/Landsat%208%20image%20processing.pdf
On page 2 we read:
“Reflectance exceeding unity: Landsat 8 also shows a few reflectance values greater than unity; usually in bright areas of snow or clouds. These values are physically acceptable. They usually arise from bright surfaces that are oriented toward the sun: that is, a sloping snow surface or cloud top slanted towards the sun. Such a surface receives more illumination per unit area than a horizontal surface and thus appears to have a higher reflectance.”
Landsat 8: Greater than 1 Reflectance Values
(The pictures show the objects for which Reflectance > 1)
http://surfaceheat.sites.yale.edu/sites/default/files/Greater%20than%201%20Reflectance%20Values.pdf
MODIS Surface Reflectance User’s Guide
(On page 10 we see that the boundary of the original data MOD09: from -100 to 16000, and the coefficient: 0.0001. I.e. it is possible to obtain Reflectance 1.6)
https://lpdaac.usgs.gov/sites/default/files/public/product_documentation/mod09_user_guide_v1.4.pdf
And other sources…
I unfortunately have not found how to consider such values in the calculations.
But it is critical in calculating various vegetation indices!
Here’s what I suggest.
Case option
if(DN>QUANTIFICATION_VALUE) rTOA=1.0
I rejected because when a large number of such pixels, we will get a distorted image. It is very incorrect
I tried to redistribute the values in the required limits and to preserve their distribution.
It is important that these values were close to 1!
The figure shows a histogram of the distribution of values for my test image (Band 2, S2A, MSI, L1C Tile), covered with snow.
And here is the histogram after my conversion – all values are within 0 - 1:
The proposed conversion.
- Determine the number of pixels with values greater QUANTIFICATION_VALUE (Light Pixels).
- Get total number of pixels = rows*cols (here taken as a generalized case without considering the background, as in the test).
- Determine the percentage of Light Pixels in the image:
double PrBadPixels = (countBadPixels100.0)/(double)(colsrows);
- We calculate the value of lower bound to QUANTIFICATION_VALUE, which will place the values of pixels with large values of QUANTIFICATION_VALUE:
double Pr = QUANTIFICATION_VALUE*PrBadPixels/100.0;
double dQ = (QUANTIFICATION_VALUE - Pr);
- The histogram computed and determined by the level of 1% of the rightmost value - the Threshold - value exceeding QUANTIFICATION_VALUE,
the number of which is small and can be neglected.
- Perform conversion of values according to the following expression (DN, QUANTIFICATION_VALUE, rTOA - float values):
if( DN < dQ) rTOA = DN / QUANTIFICATION_VALUE;
else
{
if(DN > Threshold) rTOA = 1.0;
else rTOA = (dQ + ( ( (DN - dQ)*Pr)/(Threshold-dQ) ) ) / QUANTIFICATION_VALUE;
}
To simplify you can probably reserve for these values already predetermined range from 0.9 to 1, but I have a distribution of values was not so good. Actually it caused a preliminary determination of the percentage ratio of Light Pixels.
Best regards,
Igor Garkusha
Ukraine, Dnipropetrovsk
National Mining University