Sen2three tool returns ValueError


#1

Hello all, I have tried running the latest Sen2three version 1.0.1 on my L2A products for images dated between December 2016 and January 2017, but keep getting the same “ValueError: unconverted data remains: .S” as below at all resolutions when called from both SNAP and command line.

I am wondering if it is a data compatibility problem with the latest sentinel 2 data product format or some configuration in the various python scripts. Would anyone please advise on how to address this. Much thanks.


Sen2Three cannot process Level 2A data in PSD 14.2 format from SciHub
#2

Hi,
try to install version 1.0.1 http://step.esa.int/main/third-party-plugins-2/sen2three/

Best wishes


#3

Hello,
Thanks for suggesting that, but I am using version 1.0.1 and getting the
error. I tried version 1.0.0 just to test if I get something different but
it’s exactly the same problem. Could something else be the issue? Much
thanks.

Kind regards


#4

Hello,
did you perform befeore atmospheric correction using sen2cor?

Regards


#5

Hey,

Yes post L2A processing with sen2cor 2.3.1, and with several L2A products
within the same source folder.

Kind regards


#6

Hey again,

I wondered if the problem lies within the provision for min_time and max_time acquisition dates for the products as in the image below ( as I am working with images both within and out of the provided window). I have tried concentrating on the December images just, but the result is the same error.

Kind regards


#7

Hi,
the problem is in Naming Convention of mainly folder and DATASTRIP folder and with files which are inside (im sure that with this). When I process Granules with old Naming Conventions (before 2016-12-06) everything is OK but with new convention I have the same error like above. I try to change something but any results

Regards


#8

Hi,

I think that the current version does not support the new format. From the Sen2Three Release Note (http://step.esa.int/thirdparties/sen2three/1.0.1/[L3-SRN]%20TBS2-L3-SRN%20[1.0.1].pdf):

I think that there is a minor error in the note and it refers to Sen2Three, not Sen2Cor.

Regards.


#9

Hey,

Thanks for pointing that out! Will be on the lookout for the more
compatible version. Much appreciated!!

Kind regards


#10

Hi,
it’s very funny because files with old Name Conventions is performing very well but I can’t open them in SNAP. I try to read metadata (xml) file, SNAP is doing something but doesn’t load. I can only load single band but don’t completely images and I can’t make RGB compositions Sen2Three version 1.0.1. BadumTss.

Regards


#11

Have you solved? I have your same problem!!

Can you help me?

kind regards
Filippo


#12

Hello,
unfortunately no. I think it will be solve in next release Sen2Three version. You can open image after Sen2Three in other Gis Softweare eg QGiS. The problem with opening images can be caused by S2 images version. S2 images with new naming convention are in version 14.02 or something but older images with prior naming convention (so long) are in version 13… or something.

Best Regards
Wojtek


#13

Hello all,

Can I check if anyone know when Sen2three version 1.1 will be out? I am currently trying to process 2017 data and I have stopped at the processes for Sen2cor as the naming convention has changed since late 2016 and I cannot seem to run them through Sen2three.

Does anyone has any other solutions for me to replace the “bad” pixels with the “good” ones?

Thank you so much.


#14

Hi

You can find version 1.1.0 on:

Best regards
Ferdinand


#15

Dear Ferdinand,

Thank you for the link. I have been watching the ESA webpage for Sen2three but there have not been any updates. Is there a reason why it is now called sen2proc?

Also, I am trying to install the new Sen2three now. However, I am unable to build the Sen2Three HTML Help.


I have cd to the folder of documentation in the zip file I downloaded in the link you gave and typed in “make html”. However, it gives an error that “make” is not recognised as an internal or external command, operable program or batch file.

Please let me know how to solve it. Also, I suppose the program can still run without the HTML help?

Thank you very much.


#16

I do not know why it is called sen2proc, but i think it will eventually be uploaded at the officially ESA webpage for Sen2Three, either this version or newer, probably with generated documentation as it is today.

Regarding the creation of the documentation, i did not perform this step, as it did not work for me neither. I think you need C or C++ to perform an “make”, but i am not totally sure?

As long you manage to perform the installation with: python setup.py install after building the distribution package you are good to go.

Ferdinand


#17

Hi Ferdinand et al.

some clarifications:
Sen2Proc (for seninel2 processors) is used, as it will in future contain the source code for Sen2Cor as well (alt least the open available packages).

  1. this version under sen2proc is until now not officially released by ESA, this is the reason why there is no announcement on the ESA Step website and also no On-Line help available.
  2. there is no need to compile the documentation on your own. This is only for developers, if they want to change the documentation. If you want to read the documentation, simply follow the procedure: Unpack the Online Documentation. It is completely available in html and should be readable with any browser.

Regards, umwilm


#18

I just tried to run sen2three for the first time so I am not sure I have the same format problem or a problem of installation
I get this error:

/Software/sen2three-1.0.1$ L3_Process /media/julien/NuDrive1/Consulting/Chris/Latitude\ hill/L2/
Traceback (most recent call last):
  File "/home/julien/anaconda2/bin/L3_Process", line 11, in <module>
    load_entry_point('sen2three==1.0.1', 'console_scripts', 'L3_Process')()
  File "/home/julien/anaconda2/lib/python2.7/site-packages/sen2three-1.0.1-py2.7.egg/sen2three/L3_Process.py", line 247, in main
    config.init(processorVersion)
  File "/home/julien/anaconda2/lib/python2.7/site-packages/sen2three-1.0.1-py2.7.egg/sen2three/L3_Config.py", line 1240, in init
    self.setTimeEstimation(self.resolution)
  File "/home/julien/anaconda2/lib/python2.7/site-packages/sen2three-1.0.1-py2.7.egg/sen2three/L3_Config.py", line 1258, in setTimeEstimation
    factor = self.getNrTilesToProcess()
  File "/home/julien/anaconda2/lib/python2.7/site-packages/sen2three-1.0.1-py2.7.egg/sen2three/L3_Config.py", line 1290, in getNrTilesToProcess
    if not self.checkTimeRange(L2A_UP_ID):
  File "/home/julien/anaconda2/lib/python2.7/site-packages/sen2three-1.0.1-py2.7.egg/sen2three/L3_Config.py", line 1395, in checkTimeRange
    prdMinTime = time.mktime(datetime.strptime(prdMinTimeS,'%Y%m%dT%H%M%S').timetuple())
  File "/home/julien/anaconda2/lib/python2.7/_strptime.py", line 332, in _strptime
    (data_string, format))
ValueError: time data '170601T013301.S' does not match format '%Y%m%dT%H%M%S'

Is that part of the same problem of format?

Cheers


#19

Hi. I got a similar error:

> File "/opt/conda/bin/L3_Process", line 11, in <module>
>     load_entry_point('sen2three==1.1.0', 'console_scripts', 'L3_Process')()
>   File "/opt/conda/lib/python2.7/site-packages/sen2three-1.1.0-py2.7.egg/sen2three/L3_Process.py", line 265, in main
>     processor = doTheLoop(config)
>   File "/opt/conda/lib/python2.7/site-packages/sen2three-1.1.0-py2.7.egg/sen2three/L3_Process.py", line 202, in doTheLoop
>     proc, result = processor.process(tables)
>   File "/opt/conda/lib/python2.7/site-packages/sen2three-1.1.0-py2.7.egg/sen2three/L3_Process.py", line 102, in process
>     if(self.l3Synthesis.process(self._tables) == False):
>   File "/opt/conda/lib/python2.7/site-packages/sen2three-1.1.0-py2.7.egg/sen2three/L3_Synthesis.py", line 642, in process
>     self.preProcessing()
>   File "/opt/conda/lib/python2.7/site-packages/sen2three-1.1.0-py2.7.egg/sen2three/L3_Synthesis.py", line 588, in preProcessing
>     self.setPixelMasks()
>   File "/opt/conda/lib/python2.7/site-packages/sen2three-1.1.0-py2.7.egg/sen2three/L3_Synthesis.py", line 199, in setPixelMasks
>     if self.isMoreRecent():
>   File "/opt/conda/lib/python2.7/site-packages/sen2three-1.1.0-py2.7.egg/sen2three/L3_Synthesis.py", line 267, in isMoreRecent
>     prdMinTime = time.mktime(datetime.strptime(prdMinTimeS,'%Y%m%dT%H%M%S').timetuple())
>   File "/opt/conda/lib/python2.7/_strptime.py", line 332, in _strptime
>     (data_string, format))
> ValueError: time data '170705T105026.S' does not match format '%Y%m%dT%H%M%S'

This seems to be a parsing error where sen2three attempts to read a date from a SAFE directory name.
In my case, the bug was specific to use of “MOST_RECENT” algorithm, in L3_synthesis.py – function isMoreRecent()

I was also able to avoid the problem by means of a dirty hack, simply changing the hard-code indexes. This worked for my specific product, but will likely not work for other variations of the SAFE format

# prdMinTimeS = timestamp[47:62] prdMinTimeS = timestamp[45:60]


#20

Hi all,

I encountered similar errors when I change the Max_Time parameter of L3_GIPP.xml.:

It’s perfectly working with the default Max_Time which is ‘2017-12-31’. But when I adjust it to it can run 2018 products, the above error appears.

Any thoughts?

Thanks!

-Nando