SpectralDiversityOp - estimateOffset (Error Message)

I’ve been testing today’s version of the s1tbx and I’m getting the following error when applying the SpectralDiversityOp operator:

estimateOffset: Arrays of the same length are expected.
Exception in thread "Thread-33" org.esa.snap.core.gpf.OperatorException: estimateOffset: Arrays of the same length are expected.
        at org.esa.snap.engine_utilities.gpf.OperatorUtils.catchOperatorException(OperatorUtils.java:417)
        at org.esa.s1tbx.sentinel1.gpf.SpectralDiversityOp$2.run(SpectralDiversityOp.java:618)

These are the products I’m using:

  1. S1A_IW_SLC__1SSV_20150831T005136_20150831T005204_007500_00A5B5_F33E
  2. S1A_IW_SLC__1SSV_20150807T005134_20150807T005202_007150_009C22_71C5

Regards,

Esteban
SkyGeo

Hi Esteban,

We have processed the data that you used for the test. Unfortunately we cannot repeat the problem the you have encountered. Below is the graph we have used for the processing:

Could you provide some details of your testing?

Thanks,
Jun

Hi Jun,

Thanks for helping out. Here’s the xml file of the graph.

graph.xml (12.6 KB)

Hi Jun,

I got it to work by changing the cache size options.

My previous call to gpt looked like:

gpt graph.xml -c 32G -q 12

I removed the -c 32G option and the default one (512MB?) seems to work.

I have 128 GB of RAM and 12 processors. What settings would you suggest?

Note that my /opt/snap/bin/gpt.vmoptions file includes:

-Xmx100G
-XX:-UseConcMarkSweepGC

Many thanks,

Esteban
SkyGeo

Hi, maybe I celebrated too early; it’s still processing. Maybe that’s because the cache size is much smaller now… I’ll keep you posted.

Hi @junlu,

Did you get a chance to run the graph I sent across?

Thanks,

Esteban

Hi @junlu,

I used your graph and it works perfectly. My graph doesn’t though (I still get the same error).

My current workaround is then to use a smaller graph to process the subswaths separately. Then I can merge them using another small graph.

Do you think the framework is failing to handle the tiling feature properly?

Regards,

Esteban

Hi Jun,

After further testing using your graph, I’ve managed to replicate the problem by using subswath 2 in the split op. I had only been testing with subswath 1 and so I missed it.

For example, if you choose 20150807 as master, this is a screenshot of the slave’s intensity (20150831) after running your graph:

Can you see the “nodata” strip on the right? Maybe that’s causing the problem in the SpectralDiversityOp operator.

Please keep in mind that I’m using these products:

  1. S1A_IW_SLC__1SSV_20150831T005136_20150831T005204_007500_00A5B5_F33E
  2. S1A_IW_SLC__1SSV_20150807T005134_20150807T005202_007150_009C22_71C5

Regards,

Esteban

Hi Esteban,

Thank you for your updates. I have run your graph with your test data (20150807 as master and 20150831 as slave) and did not have any problem. But I ran it only with the first two bursts of subswath 2 in order to shorten the processing time. The data in the “nodata” strip on the right of the image is not used in SpectralDiversityOp. The garbage in the “nodata” strip in the output of SpectralDiversityOp has been fixed although I don’t think it causes any problem. By the way, it probably more efficient to process each subswath independently with one graph than processing 3 subswathes simultaneously in one big graph.

Regards,
Jun

Hi Jun,

Thanks for the tip. I’ve tried to process it again with version 3.0.0 and I’m getting this:

I’ve taken your advice and so I’m processing one subswath at a time. The only difference is that I’m using the ProductSet-Reader and SliceAssembly ops two read these products:

Date: 20150807 (master)

  1. S1A_IW_SLC__1SSV_20150807T005110_20150807T005137_007150_009C22_E118.zip
  2. S1A_IW_SLC__1SSV_20150807T005134_20150807T005202_007150_009C22_71C5.zip

Date: 20150527

  1. S1A_IW_SLC__1SSV_20150527T005106_20150527T005133_006100_007E93_C1ED.zip
  2. S1A_IW_SLC__1SSV_20150527T005132_20150527T005159_006100_007E93_969F.zip

Have you seen this before? I’ve also tried with another slave (see below) but that one goes fine.

Date: 20150503

  1. S1A_IW_SLC__1SSV_20150503T005105_20150503T005132_005750_007624_81BC.zip
  2. S1A_IW_SLC__1SSV_20150503T005130_20150503T005158_005750_007624_54E8.zip

Regards,

Esteban

For the sake of clarity: I did check that the SliceAssembly op applied to the 20150807 and 20150527 products works fine. These artefacts appear after coregistration.

Hi Esteban,

Can you be more specific on your processing chain? At which step did you apply SliceAssembly? Generally we don’t suggest users do SliceAssembly before Backgeocoding. SliceAssembly can be applied at later stage, say after deburst.

Regards,
Jun

Hi Jun,

Sure, here’s a screenshot of the processing graph:

The reason why I use the ProductSet-Reader andSliceAssembly ops in my processing chain, is that products don’t always fully overlap (see my conversation with @lveci here). So, I assemble them first and then select bursts in the TOPSARSplit op using the wktAoi parameter.

Incidentally, this is a big issue with the stacks over the Netherlands, too.

Please let me know and thanks for taking the time.

Esteban

Hi Esteban,

I think the problem is in product S1A_IW_SLC__1SSV_20150807T005134_20150807T005202_007150_009C22_71C5.zip (Slice#3). The geolocation grid of this product could be wrong. I think the geolocation of the Nth burst should actually be the geolocation for the (N-1)th burst.

This can be seen when you do the slice assembly of this product with product S1A_IW_SLC__1SSV_20150807T005110_20150807T005137_007150_009C22_E118.zip (Slice#2), The azimuth time and geolocation of the first burst of Slice#3 should be consecutive to that of the last burst of Slice#2. Check the metadata in both products, you will find that it is true for azimuth time, but not true for geolocation. The geolocations of the two bursts are not consecutive, they are actually overlap.

Because of this problem, you cannot coregister the following products
S1A_IW_SLC__1SSV_20150807T005134_20150807T005202_007150_009C22_71C5.zip
S1A_IW_SLC__1SSV_20150527T005132_20150527T005159_006100_007E93_969F.zip

Hope this found helps,
Jun

Hi Jun,

Thanks for the quick reply. That’s a good catch!

I’d have two questions:

  1. How do you suggest I proceed with these invalid products. Should we have ESA redeliver these? or can the geolocation be fixed by a S1TBX operator?
  2. Do you think my graph can be added as a standard graph in the toolbox? As discussed in this topic, having products with limited overlap is a recurrent issue with S1 data, which defeats the purpose of having S1 for deformation monitoring using stacks of SLCs.

Regards,

Esteban

Hi Esteban,

I think you can coregister
S1A_IW_SLC__1SSV_20150807T005134_20150807T005202_007150_009C22_71C5.zip (burst 2 - 10)
S1A_IW_SLC__1SSV_20150527T005132_20150527T005159_006100_007E93_969F.zip (burst 1 - 9)
first with the following graph

Then coregister the other pair
S1A_IW_SLC__1SSV_20150807T005110_20150807T005137_007150_009C22_E118.zip (burst 1 - 9)
S1A_IW_SLC__1SSV_20150527T005106_20150527T005133_006100_007E93_C1ED.zip(burst 1 - 9)
with the same graph.

Finally run TOPS Slice Assembly to the two coregistered images, and run TOPS Deburst to the assembled image. That should give what you want.

Regards,
Jun

Hi Jun,

Thanks for the workaround. I’ll try that.

Question: is this the right place to report what I mentioned in this post?

Please let me know.

Regards,

Esteban

Hi Esteban,

No, I think they should be addressed in their original post.

Thanks,
Jun