Advice and Experience of SNAP on Cloud Computing platforms

RSS CloudToolbox service “provides registered and authorised ESA EO-SSO users with customised virtual machines (VMs) made available on a cloud infrastructure.” This forum has a user’s efforts to configure the SNAP Python module .

At my work we have quite a number of 24 core machines (two Intel E5-2630v3) processors, but a single disk for data. These perform very well on some numerical models, but for batch processing with NASA OCSSW and ESA GPT many of the jobs take much longer than on a garden variety core i5 system. Many factors may be involved: a) as I recall, the E5-2630v3 processor has one floating point unit for 3 cores, b) the single disk is a bottleneck, c) it is difficult to ensure that cache memory is used effectively, and d) the kernel scheduler (for Ubuntu 14.04) may not be indeal for this combination of workload and processor. Now we are promised mitigation for Meltdown and Spectre that may affect performance.

Since it can be tricky to get good performance from multi-core systems, it is worth spending a bit of time looking for “easy” fixes such a as upgrading the disk storage for better I/O thruput and doing some experiments with process affinity.

Thre are other advantages of cloud systems: someone else is responsible for backups and maintenance of hardware and software.

2 Likes