Processing Sentinel 1 from AWS Open Data

Microwave Toolbox
#1

Hi Folks,

My current project is moving an on-premise Sentinel 1 processing pipeline onto AWS.

Part of our migration is aiming to use the Sentinel 1 Open Data Bucket https://registry.opendata.aws/sentinel-1/. The main reason is that S3 -> AWS will reduce download speeds, another benefit is that I believe the SciHub data is now a rolling archive.

Does anyone have experience processing this dataset with SNAP? We’re planning on running Subsetting, Radiometric Calibration, and Terrain Correction.

Any insights are hugely appreciated!

Ciaran

1 Like
#2

@ABraun @marpet

Do you folks have any idea if this is possible?

When I open a tiff, for example: GRD/2019/1/1/IW/DV/S1A_IW_GRDH_1SDV_20190101T003857_20190101T003922_025277_02CBAA_DDD4/measurement/iw-vh.tiff from the Open Data bucket inside SNAP and try to run Radiometric Calibration on it, I get the following error:

image

#3

SNAP needs the metadata stored in the safe file in the top level of the product. Opening the tif file alone technically works but it does not contain any information on incidence angles, sensor specifics ect which are needed for radiometric calibration.

#4

So if I was to download the manifest.safe file for the given tiff, would that potentially work?

I’m hoping to do this programmatically with snappy and use the AWS hosted data, is there a nice way I can open them ‘together’?

Thanks for the help!

#5

I’m not very experienced with snappy, but whenever I used it I open the product with the safe file like in this example: http://remote-sensing.org/preprocessing-of-sentinel-1-sar-data-via-snappy-python-module/

You will have to test how much of the other files and folders you need for a correct processing.

#6

So I know roughly how to use snappy, I guess my question should rather be:

If I have the .tiff and the manifest.safe how can I run something like Radiometric Calibration within SNAP?

Is it something trivial like zipping them together and opening that way?

I’ve just tried to open the manifest.safe for that image and I get:

image

#7

I think the overall structure of the product folder needs to be maintained. So the manifest.safe is in the top folder and the tif file is in the measurement folder.
But there is also a support folder which contains files like S1_level1_calibration.xsd, so I assume they are needed as well.

Maybe I missed the point: Why are you not downloading the entire product folder from AWS?

#8

I assumed I could just use the band I needed, the folder is not a ‘single’ downloadable on AWS, it’s merely single files separated by keys.

I will download the entirety and let you know how it goes!

#9

I see. Maybe you can progressively remove parts of the product folder and check if it works.

#10

Hey, thanks for the help again.

Turns out it was a easy as just downloading the entirety of that prefix and then SNAP happily opens it as a Sentinel 1 Product.

Not the worst thing in the world as we can just clear up the downloaded files :slight_smile:

Thanks again!

2 Likes
#11

Hi,

Turns out it was a simple as downloading the total of that prefix and afterward SNAP joyfully opens it as a Sentinel 1 Product.

Not the most exceedingly terrible thing on the planet as we can simply clear up the downloaded documents
I accepted I could simply utilize the band I really wanted, the envelope is certainly not a ‘solitary’ downloadable on AWS, it’s only single documents isolated by keys.So if I somehow happened to download the manifest.safe record for the given spat, could that conceivably work?

I’m wanting to do this automatically with smart and utilize the AWS facilitated information, would there say there is a great way I can open them ‘together’?