I am a SeaDAS 7.5.3 user (Ubuntu 18.04 lts) for MODIS data.
I am extracting 1000 pixels from a list of dim files. Because I have to do more than two hundred extractions, opted to do that trough gpt within a loop for batch processing.
However, I realized that if I PixEx trough GUI from SeaDAS, it generates a file with more than double the size of the file generated from gpt with xml file. I generated the xml file from GUI for comparison but could not detect the difference. I’m affraid my batch processing might be missing something that causing less extraction than I need.
I imported the same file of 1000 points into SeaDAS GUI for PixEx and generated the xml file and attached it here. I copied the dim files into a exclusive folder to make the GUI extraction easier.
The gpt, xml and par files for the batch processing are compacted into one file and attached as well. The other attached file has the 1000 points needed for both methods.
Would you please have a look into my two ways of PixEx? I want to know how to correct the batch processing for PixEx to extract correctly.
It would be helpful to know more about the differences between the two output files. Do you have the same memory settings for seadas.vmopts and in
gpt.sh? Have you tried the same extractions using ESA SNAP?
We can’t reproduce your problem because we don’t have your data files. I use PixEx routinely (but mostly the SNAP version) and have not encountered differences between GPT and the GUI.
You should try to make a small example that is easy for others to reproduce. Do you see differences when using a single dim file?
Hi.
Thank you for your reply.
The differences are not only in the size of the files with extracted pixels only but also the pixels where the data are extracted. For some reason (I guess there must be in the .par or .xml files), gpt.sh does not extract some pixels.
I used to use GUI only… just now when I need to PixEx 1000 points from a bunch of files, started to use gpt.sh and its .par / .xml files. Very fresh here
I found out since my post earlier that there are some differences in the .xml informations from GUI and the .xml that I feed the gpt. The size of the files are closer now but gpt.sh is still extracting less pixels than GUI. I plot the 1000 points in GUI and I can see there are data in some points but gpt does not extract some of them.
How can I check the memory for seadas and gpt? I don’t have the SNAP version either.
I am extracting the 1000 points from 9 dim files and those files are too big to attach them here.
Would you suggest me anything, please?
My best regards.
Claudia
Are the points that are missing from the gpt results missing for all files or do the missing points vary across images? To examine multiple PixEx results it helps to combine them into a single file with the command-line join program (I first edit the headers to one line with the names of each column). If you can find a pattern in the missing data you may be able to construct an example others can use. You can also try running two GPT jobs, each using half of the data to see if that brings in the missing points.
One possibility is that gpt runs low on memory and this triggers a bug that is not usually seen on systems that use more memory. You can add the “-e” parameter to the gpt command line to see if you get error messages. Both the GUI and GPT are Java programs that run in memory set when Java starts. There are multiple types of memory and associated parameters, but the one that usually needs adjusting is “-Xmx”.
The Xmx value for seadas is set in <install-dir>/bin/seadas.vmoptions>.
You can try a value up to 2/3 of the system memory. For gtp the value
is set in the gtp script (.bat on Windows, .sh on linux). These are ASCII files and can be edited with most text editors.
If you can’t find the problem without more help, you need to construct a much smaller example that demonstrates the problem so you can upload everything needed to reproduce the problem.
