Hello @nuno.miranda Could you check the FTP account for us please
FYI, I am creating a public repository at https://github.com/jmfriedt/sentinel1_level0 with my current investigations of Sentinel1 Level0 decoding software. In addition to using some raw datasets, I check on the reference dataset mentioned at the end of https://sentinels.copernicus.eu/documents/247904/685163/Sentinel-1-Level-0-Data-Decoding-Package.pdf (section 5.3) that decoding is working properly, although I have not figured out yet how to compare with the TIF outputs. No idea yet how far this is going to get …
1 Like
I am struggling with FDBAQ decoding and am wondering if I am reading correctly the binary file.
1/ I am confused with BAQ mode page 33 of SAR Space Protocol Data Unit. The documentation says that BAQ mode is bits 3 to 7 of byte 37 of the secondary header, and reading this byte value I always get 0x0C. The confusing part is that 0x0C has not been shifted >>3 to remove the error flag and n/a. If I shift I get 1 which is “not applicable”, while 0x0C prior to shifting would match nominal “FDBAQ” but why the value without shifting ? I believe I am reading the header correctly as the Space Packet Count and PRI Count just before this field increment correctly from one block to the next
2/ assuming nominal FDBAQ, I dump the User Data Field in a file for decoding prior to implementing the Huffman decoder. The S1_L0_Decoding_Package only provides the final output after Huffman decoding and application of reconstruction by applying THIDX. Would it be possible to have the intermediate step of just Huffman decoding, for example the 1st two 128-samples of S1A_IW_RAW__0SDV_20200608T101309_20200608T101341_032924_03D05A_A50C.SAFE/s1a-iw-raw-s-vv-20200608t101309-20200608t101341-032924-03d05a.dat ?
3/ just to make sure I dump the User Data Field field correctly: I observe that NQ is always 11919 in S1A_IW_RAW__0SDV_20200608T101309_20200608T101341_032924_03D05A_A50C.SAFE/s1a-iw-raw-s-vv-20200608t101309-20200608t101341-032924-03d05a.dat: is this correct ?
Thank you
Hi @jmfriedt,
This is a nice initiative you have taken. I’m not sure if the people who might be able to help you, are reading here in the forum.
Maybe @mengdahl can tell you a better point of contact.
Thank you for your support. Actually thanks to the code provided by the author of
who kindly contacted me by email, I managed to compare my code with his working example and have achieved a realistic map of my first data processing as shown at https://github.com/jmfriedt/sentinel1_level0/ This is still far from operational as the software meets an unexpected condition (impossible BRC value while processing) and stops at some point – and I am aware that only a subset of the possible cases is being implemented (FDBAQ type D) but as least some basic insight in the data structure seems to have been gained. The idea of indexing the bit number from 0 for the most significant bit (page 10 of the Packet Protocol Data Unit) strikes me as awkward at best and had me struggling for a week to understand the Huffman encoded data order, but so be it, the information is indeed documented although I had started reading from p.13 after the introductory tables of the documentation !
Thanks.
1 Like
A bit more progress as some error in BRC3 was corrected and now a whole raw dataset (S1A_IW_RAW__0SDV_20210112T173201_20210112T173234_036108_043B95_7EA4.SAFE – echos only, discarding calibration at the moment) leads to consistent decoding. BRC4 was added to the software but not validated since the dataset I am testing on does not require this encoding – will exhibit some flaw in the state machine most certainly when tested. Plotting the raw dataset magnitude seems consistent (tentatively interpreted as 5 bursts of one swath). As I am not familiar (yet) with range compression and even less with azimuth compression I have not validated the comparison with the processed data, but at least some files to play with if needed.
Back to basics: a complete StripMap dataset was processed, demonstrating I believe proper decoding of the raw data, conversion to {I,Q} coefficients and chirp shape estimate from telemetry. Range compression is achieved by only cross-correlating the predicted chirp shape leading to a single pixel wide correlation peak for point-like targets, which is fine. However I am confused with azimuth compression. Looking at the bottom of https://github. com/jmfriedt/sentinel1_level0 (Sao Paolo StripMap processing) we see that the phase along azimuth is parabola shaped meaning a linear frequency shift which I did not expect since in Ground Based Synthetic Aperture Radar (GB-SAR) the phase is expected to be linear with a slope equal to the spacing interval between measurements. Further analysis (https://github.com/jmfriedt/sentinel1_level0/blob/main/figures/compression.png) allows for recording the pulse shape along azimuth following range compression and indeed applying(cross-correlating) this pulse shape along the azimuth direction to to the whole image as shown on https://github.com/jmfriedt/sentinel1_level0compression3.png seems to achieve azimuth compression on the whole image with a single pixel wide cross-correlation peak. Could anyone hint at the cause of the leftover linear frequency modulation after range compression and how to identify the pulse shape along azimuth ? This is for SM and not IW/EW where I understand that some leftover frequency shift is expected from beam sweeping along the swath. I have the feeling this is related to Doppler centroid identification but at the moment this topic is beyond my understanding. Any pointer towards relevant information would be welcome. Thank you.
Apologies for the spaces in the github url … the Step forum won’t allow me to post this comment/question otherwise.
1 Like
I tried to fix the figure urls, but somehow could not find a target, can you please try again to post them? Otherwise, you can send them to me as a private message and I will insert them.
Concluding (at the moment at least) this whole investigation, I modified the software to output binary files, one for each swath, named with the NQ number of samples/range and the GPS time of the acquisition. To demonstrate the integrity of the generated datasets, I compare my output at https://github.com/jmfriedt/sentinel1_level0/blob/main/figures/saopaolo_SM_level0.png with the output of SNAP analyzing the Level 1 data of this StripMap dataset at https://github.com/jmfriedt/sentinel1_level0/blob/main/figures/saopaolo_SM_snap.png and I believe the results are close enough to demonstrate the proper decoding of the Level 0 data. At the moment I am not providing the satellite attitude data (quite easy to add from the current software architecture) nor is the processing for range and azimuth compression as fancy as those provided in the ESA documentation. Range compression uses a correlation with the nominal chirp shape and once achieved, I search for the strongest echo along the azimuth and assume this is due to a point like source and hence this echo is the impulse response of the chirp. I correlate all azimuth with this chirp which I have not (yet) understood how to construct analytically, without applying Doppler centroid analysis (best explained imho in Madsen, S. N. (1989). Estimating the Doppler centroid of SAR data. IEEE Transactions on Aerospace and
Electronic Systems, 25(2), 134-140. https://doi.org/10.1109/7.18675 which includes a core information I have not found in the ESA documentation, namely that the autocorrelation quickly vanishes to 0 and that only samples at delays +1 or -1 are relevant, as observed experimentally). Since my initial objective was EMI analysis, I think I am going to stop there unless more features are requested. Thank you ESA for providing this amazing datasets and the documentation needed for processing, that was fun.
1 Like
Since my original purpose in this whole endeavor was recording ground-based emissions detected as RadioFrequency Interferences as described at https://www.mdpi.com/2072-4292/9/11/1183 during the RX channel calibration sequence, I have completed the available software with Data Format A (Bypass) calibration sentence decoding. The results seem quite consistent as shown at https://github.com/jmfriedt/sentinel1_level0/blob/main/calibration_S6.png but still require interpretation.
Radiometric analysis of the rank echoes at the beginning of each IW swath records:
image: path508.png
A “short” video for a presentation at the European GNU Radio Days 2021 summarizing the latest investigations on Sentinel1 Level 0, including preliminary attempts at passive RADAR using Sentinel1 illumination: http://jmfriedt.free.fr/gnuradiodays_setinel1_level0.mp4 with slides at http://jmfriedt.free.fr/slides_sentinel_raw.pdf
Hello, Nuno
I have been trying to make use of S-1 Level-0 data decoding package that you helpfully provided. The problem is interpret data found at https://nas-ext.cls.fr/sharing/jybkQ83QE , as referenced in the pdf document. The question is about TIFF_000[1-8] files found
in S1A_IW_RAW__0SDV_20200608T101309_20200608T101341_032924_03D05A_A50C.TIFF folder . The libtiff-based tools do not recognize those as tiff-files,complaining about bad
magic number. What is their format?
I am definitively not an expert on any of this but perhaps those files are bigtiff? This document seems to pass that idea…
Perhaps they are, but the tools that I’ve tried do not recognize those files as any kind of TIFF.
Hello, I need the current link to download the two reference products described in the document “Sentinel-1 Level-0 Data Decoding Package” provided to support the decoding of Level-0 products of sentinel 1. I am working on Raw sentinel 1, I will use SNAP 9.0.0 to open the tiff file produce for comparison the file mentioned in the document are listed below. The link nas-ext.cls.fr_sharing_jybkQ83QE is no longer working. @nuno.miranda please help
Level-0 in Sentinel-1 format:
o S1A_IW_RAW__0SDV_20200608T101309_20200608T101341_032924_03D05A_A5
0C.SAFE
• RAW decoded:
o S1A_IW_RAW__0SDV_20200608T101309_20200608T101341_032924_03D05A_A5
0C.TIFF
Hi ,
Just to let you know that avalentino developped a (almost) full python package to decode the S1 ISP.
The repo can be found here.
This would remove the dependence from any reference L0 pre-decoded data as the s1isp could generate
it from any L0.
External contributions to this package is welcome.
Cheers,
Nuno & Antonio
2 Likes
Thanks a lot