I tried this again using a rather big (and expensive) compute node in AWS to run this. It was a c4.8xlarge, a compute optimized node with 16 cores, 30 GB of memory, and ran it like this:
./gpt myGraph.xml -q 128
… and got similar results. I’m not sure when it goes into a single core, but eventually it does (after many hours) and never comes back to fully parallelized. If I add -c parameter, I almost immediately get a java heap space error so I’ve left that alone. But it never really seems to finish what it’s doing. I kick it off using screen, since it runs so long it drops my connection to the remote host after a few hours. And when I reattach to screen , it’s still processing. I check the output file and a full day later it has made no changes, but the file is junk.
Here’s my graph, roughly following @con20or’s post here about Oil Spill:
read > calibration (using VV only) > land-sea-mask (sigma0_vv) > oil spill detection (sigma0_vv) > oil spill cluster > write
I’m using the smaller window size of 61, threshold of 2.0, and small cluster size of 0.1 just to get it to run.
Any suggestions appreciated!
-CS