A way to make the processings faster?

Hello,

I am currently processing Sentinel 1 images, both in GRD and SLC, and I wonder if there is a way to make it faster. The setup of the machine that I use for most of the processings has 40 Gb of RAM, and I am using .xml graph integrated in python (with subprocess.check_call, and the GPTcmd).

On average it takes 3-4 hours to process an SLC image (Coregistration>Interferogram>Deburst>Merge>TopographicPhaseRemoval>TerrainCorrection>SpeckleFilter) and 40 min to process a GRD image (Subset>ThermalNoiseRemoval>ApplyOrbitFile>Calibration>TerrainCorrection>SpeckleFilter>LinearToFromdB>BandMaths(x2)>BandMerge)

I hesitate to upgrade to 64Gb of RAM, but would that really increase the speed of my processings ? Also I am wondering if python or the snap GPTcmd limits the use of RAM and of the processors ?

Thanks in advance (and sorry for my poor english)

have a look here: Snap works very slow

most of the time, upgrading to SSD is the best option to start with, before adding RAM

Thank you for your response, I just requested a virtual machine from the RUS service.

Otherwise I read that it is possible to modify the RAM that snap can use through the snap-conf-optimiser. I was wondering what is the unit for the -Xmx and the -Xms (right now it has -Xmx27648m and -Xms256m) ? For example how much should it be so that snap uses 30Gb of RAM out of the 40 that the machine has ? For the stack process that I am currently running, I am not sure that it uses everything that it could :

I’m not saying that this will fix your problem, but I’ve seen SNAP getting installed with a too large max heap size (e.g. 90 GB on a 128 GB machine), and reducing it (-Xmx) to something more reasonable like 8 or 12 GB actually sped up the SAR processing chain I was looking at. It might be worth trying, although from your screenshot it doesn’t seem to be the case.

But if it works, you can process two products at a time.

Another idea: set up a RAM drive (there is some software for that) and copy the input product to it.

Third idea: try with a different output format like BEAM-DIMAP, it’s possible that some of the writers are slower than others.

2 Likes