How to decode Sentinel-A L0 raw data? how to accomplish range compression correctly?

I just wrote the decoding code based on the Level-0 products of Sentinel-1A, according to the library documents downloaded from the website: earth.esa.int, named “A6_S1-IF-ASD-PL-0007_SAR Space Packet Protocol Data Unit.pdf”. I have 3 questions to ask for help:

  1. I tested several IW mode data(TOPS), and I found that statistical property of echo I decoded is abnormal (shown in Figure below), only focusing on several values(0, 13, 128, 245). I am wondering what’s the matter, whether the decoding of BAQ or statistical approach?
  2. I did the range compression using the echo I decoded, based on the first burst. The parameters I used are: sampling rate is 6.434523812571428e7 Hz, bandwidth is 56.5MHz, pulse width is 5.240481033595628e-5, but the compression result is incorrect. I don’t know where the problem is, is it caused by compression method or echo decoding method ?
  3. Is there a processing tool based on the Sentinel-A raw data ? Covert raw data to complex echo data.

1 Like

Hi,

The results are pretty strange indeed. I can confirm that several other people have used successfully this datasert to develop their own decoder. In other words, it is not a compression issue but a decoding issue, I am afraid.

I provided in addition to RAW data a version decoded such that you can verify pixel by pixel the result of your decoding.

The FDBAQ decompression is pretty complex. I suggest you carefully review the implementation of the section 4 of the S-1 SAR packet protocol data unit (S1-IF-ASD-PL-0007).

The simple fact of using correctly the tables provided in section 5.2 makes such that the Svalue is a float number and not an integer. This is another indication of something wrong in the decoding.

Kind regards
Nuno Miranda

Thanks for your reply. @nuno.miranda. Maybe I didn’t express my statistical approach cleraly. The interger value shown in the picture was the result of quantization among 0~255. The decoding result is float type.

You said that you provided a version decoded. How can I get it ? Could you give me a sample result? I want to learn about it and verify my result. My email is elivesyy@gmail.com.

Eagerly awaiting your reply. Thank you!

Thanks
Best regards
Alfred

All the information you need is on this document (available in the library)
Sentinel-1 Level-0 Data Decoding Package (COPE-GSEG-EOPG-TN-15-0005, 28/01/2015 ). Section 5 and 6 desribes the test data (RAW and decoded)

I got it. Thanks very much! @nuno.miranda:slight_smile:

It would be great to have the decoding-software in the public domain… :wink:

Dear Alfred,

The document I was referrring to was in the document library. Since the Jan 2015.
Wiht your last remark, I understood that you have some difficulties in finding. After checking, I realise that actually it has been removed. I can’t tell when or why.

I asked to put it back in the library and it is there (as free and open as one could wish.) and here it is:

https://sentinels.copernicus.eu/web/sentinel/user-guides/sentinel-1-sar/document-library/-/asset_publisher/1dO7RF5fJMbd/content/sentinel-1-level-0-data-decoding-package

I apologise for the inconvenience and I am sure the info in their will help you to find your issue.

Cheers,
Nuno

Hi all
I’m trying to decode L0 by myself.
I get some abnormal values of ‘PulseStartFrequency’ parameter on some IW products. Like ~4.8GHz for IW1&IW2 and ~7GHz for IW3
Centre frequency is said to be 5.405 GHz, but what is operational range? Could’n find it anythere

Thanks a lot !

This is my IDL code.
It is the exact formula as what is written in the document section 3.2.5.7

;+
; :Description:
;    Describe the of the Tx Pulse Starting Frequency in [MHz]
;
; :Params:
;    txpsfCode : code of the TXPSF. The code shall be signed already
;    txprr     : value of TX Pulse Ramp Rate in [MHz / us] 
; :Author: nunomiranda
;-
function decode_txpsf, txpsfCode,  txprr, HERTZ=HERTZ
  fref =37.5347222D ;[MHz]
  
  txpsf = (txprr / 4D / fref) +  txpsfCode * fref / 2D^14
  if keyword_set(HERTZ) then txpsf*=1e6
  return, txpsf ;
end

Also the S-1 level-1 product SLC or GRD report the values decoded by the processor. For example:

<downlinkValues>
          <pri>5.823674372819869e-04</pri>
          <rank>9</rank>
          <dataTakeId>10144</dataTakeId>
          <eccNumber>8</eccNumber>
          <rxChannelId>1</rxChannelId>
          <instrumentConfigId>1</instrumentConfigId>
          <dataFormat>
            <baqBlockLength>256</baqBlockLength>
            <echoFormat>FDBAQ</echoFormat>
            <noiseFormat>BAQ 5 Bit</noiseFormat>
            <calibrationFormat>Decimation</calibrationFormat>
            <meanBitRate>1.321665316099207e+00</meanBitRate>
          </dataFormat>
          <rangeDecimation>
            <decimationFilterBandwidth>5.659000000000000e+07</decimationFilterBandwidth>
            <samplingFrequencyAfterDecimation>6.434523812571429e+07</samplingFrequencyAfterDecimation>
            <filterLength>36</filterLength>
          </rangeDecimation>
          <rxGain>-4.000000000000000e+00</rxGain>
          <txPulseLength>5.240481033595628e-05</txPulseLength>
          <txPulseStartFrequency>-2.825153419637256e+07</txPulseStartFrequency>
          <txPulseRampRate>1.078230321255894e+12</txPulseRampRate>
          <swathNumber>10</swathNumber>
          <swlList count="1">
2 Likes

could’n figure that TXPSF is not start frequency itself, but I should add it to centre frequency

I don’t believe that actually you need to add the radar frequency to it.
The txPulseLength, txPulseStartFrequency, txPulseRampRate are all what you need to construct the chirp and range compress the data
N

Dear Nuno.

I downloaded the sample data decoded you referred. I found that there were several TIFF format files. I tried to read them in the Matlab, using the function ‘imread(‘xxx.tiff’)’, but an error happened: ‘Error using rtifc. Unsupported sample format 5’. How to read this type of TIFF file and get the pixel value ?

Best ragards
Alfred

Yes, I think so. Most of current software are focusing on the processing based on the image product, not the raw echo data. It is important for many researchers who major imaging formation algorithm to use the decoded raw data. I look forward to use such decoding-software very much. :slight_smile:

At ESA, RAW data have always been considered as users product. We always made them available starting from ERS and we continue for S-1. THis is not the case for other missions. However, we never made available RAW decoding software.
We have supported and will continue supporting anyone who would like to develop a “decoder” and we fully support if this is made available in public domain.

My personal opinion is that people willing to process S-1 RAW data (or any other mission) needs to learn about the sensor specificities.
If you want to process S-1 TOPS data, you have to learn about S-1. Being able to decode the RAW data is part of the learning curve. I can witness it>

The issue you are encountering is related to the matlab TIFF support that doesn’t understand complex format pixels. The standard tiff library and also GDAL seem able to properly understand this data (including IDL+ENVI).

For example:

tiffinfo S1A_IW_RAW__0SDV_20140825T224532_20140825T224605_002102_00217A_7AF4.TIFF_0001.tiff
TIFF Directory at offset 0x8 (8)
Image Width: 19768 Image Length: 15510
Bits/Sample: 64
Sample Format: complex IEEE floating point
Compression Scheme: None
Photometric Interpretation: min-is-black
Samples/Pixel: 1
Rows/Strip: 1
Planar Configuration: single image plane

1 Like

Dear Nuno Miranda,
sorry to use this post relating to another product but maybe you can help me with my request.
For my PhD project I need to work with Level 0/RAW ERS data, collecting all possible info recorded during the acquisition step and used in the focusing stage. I have the permission from ESA to download them through Eolisa. Unfortunately the files have extension .E1 or .E2 and and not .dat (as requested in commercial software as ENVI that reads ERS-1 Level-0 with extension dat*.001).
Do you know some available software to open this type of data? Moreover, do you know if some document for decoding is available?

The native format of ERS was the CEOS format that was a folder based product containing several file like the leader file, the dat file (dat.*001).

When came ASAR into play ESA have started proposing to the users to get ERS in native or in ENvisat format. The two formats were maitained in parallel for many, many years. In the last years, ESA have put in place a huge activity aiming at consolidating the ERS archive: rapatriating the old magnetic tape from every stations in the world, dumping their content (when possible) and keeping, perform an analysis to segregate good from bad data in order to store the data with todays standard ensuring that users can still have access to this data in the future. The idea is to allow users to work on long time series ERS1 + ERS2 +ASAR +S-1.

As part of this activity, it could very well be that the decision was taken to no support the native CEOS format. I recommend to drop an email to eohelp@esa.int for a clear confirmation.

RAW is data not processed, I am not sure that s/w Like Envi will do anythoing with it (except you have the sarscape extension that may do the SAR focussing!?). The ERS in envisat format is nothing else the ERS ISP packed with with MPH/SPH acting as leader file. All the information can be found here: https://earth.esa.int/web/guest/deprecated/data-access/browse-data-products/-/article/sar-annotated-raw-data-product-1016.

If this format is not supported by ENvi/SARSCAPe you need to go back to the Envi people to update the s.w accordingly. Another solution is that you get directly the SLC or PRI from ESA.

However, I recommend to ask confirmation to eohelp in first place.
Kind regards,
Nuno Miranda

Dear Nuno,
thank you for your reply and I’m sorry to respond to you so late. Unfortunately ENVI is not able to deal with radar raw data and SARscape is an expensive tool and not useful for my project. I need to decode the RAW data to collect all the info associated to it. I read your response about S1 RAW data and I would like to learn how it is possible to decode it. Do you know some open source code to do that?

Dear Nuno,
the document Sentinel 1 SAR Instrument Calibration and Characterisation Plan (S1-PL-ASD-PL-0001) is not available for download.
Will I need it too to decode level 0 data?
Regards,
Igor

Hi Nuno,
the FTP account in this document doesn’t work anymore. I want to debug the huffman decoder in my fdbaq decompressor. Is there any chance an example file could be posted again?

1 Like