What is the expected processing time for stamps export?
I’m using the snap2stamps workflow on a full subswath of S1 data, coregistration/ifg formation completed on average just under 30 minutes but the export step is taking 3-4 hours per scene. I’m processing 98 scenes and it’s about 60% done after running for the last 10 days so it should be done sometime next week.
I was under the impression the most time consuming step was in the co-registration so I just want to check with how long the stamps export is supposed to take and what other people have seen.
It doesn’t seem to be too cpu or memory intensive so i’m assuming the bottleneck is in the disk read/write. I’m exporting it to an external hardrive (Backup Plus Portable Drive 4TB Blue) with usb 3.0.
facing the same situation with a complete subswath of S1A. Coregistration and interferometric time for one slave is about 490s, and then I am still in waiting of the first export to finish.
I would be happy to understand what happens during the export step. Obviously, it is not just a procedure of compatibility between formats, as “export” might be thought of.
It looks like it does some major calculations in that step.
@ABraun, Can you please shed some light on the subject?
I have a same experience a few days ago.
In my case, the problem is my HDD data transfer rate is low (external HDD with USB 3.0). Then, I changed my working directory to new HDD with transfer rate 6GB/s and connect with SATA (not USB). The processing speed (stamps export) using new HDD was increased 10-15 times than previous HDD.
Dear @mdelgado@ABraun InSnap2stamps (Python code) all the steps are running with high speed but Stamps_export.py is running very very slow and time-consuming (for one it is taking 3hours ) even I’m using a 128GB RAM server. Is there any method to improve the speed?
Dear @marpet and @mdelgado; while using stamps_export.py it takes more than 2.5 hours per image pair to get exported. Every previous step ran with minimum time of 1-7 minutes, but the process is running forever in this step. I have previously exported this AOI in just under 20 minutes (5 bursts). I use 20 core processor, and 96 GB RAM. I tried running it several times but without any success. It is pertinent to mention that read and write is happening on an external HDD. I have tried to export smaller areas but same time lag. What could possibly be causing it?
Well… if you had better performance before I can only ask you to check and clean as much space as possible as well as asking you to use the fastest external disk (possible SSD on USB 3.0) you can get. Otherwise this can also be affecting performance.
If you may have a faster internal storage (SSD internal) better to use any HDD external. Sometime the easiest step takes much more time just due to I/O operations.
I hope this helps, but I cannot think on another reason for such poor performance.
Best,
I have had severe internet issues. Does Stamps Export necessarily require internet connection? I have done the export for many products without internet connection and when the latter finally become stable, this message appeared in terminal during the export process.
