Are you sure the process is completed? Maybe SNAP crashes at that point and that’s way you only see 17 products. If you are using the GUI to process large datasets in batch processing you most probably will face memory issues. If you have a good machine this can be OK but 105 sentinel-1 products can fill up your computer.
Did you try GPT? That would be a good and faster solution. While the GUI interface does not release the memory when it finishes processing one input, GPT does it. Which are the specifications of your machine? Check the performance during the process to see how the resources are managed.
Thank you @marpet,
I’m not a computer programmer, so writing code in gpt mode is quite difficult for me.
I need to make the process in batch mode, that seem complicate a lot the code needed. I’m wrong?
It is not straight forward, especially for people who are not very familiar with programming. But even our GIS people here got used to it. So it is not too hard to learn.
as an example, if you want to cut out a geo-region just do:
The geo-region is given in WKT format. This you can get from a geometry you have drawn in SNAP desktop.
Of course, you can do a lot more fancy stuff, but maybe this is all you need at the moment.
Yes @marpet , but that code is for a single product. I need to do that process for 105 products. The single product process is quite easy but I need the batch mode.
I really suggest you to take some time to learn how to use GPT. It will make your work with Sentinel data much better. Start by learning how to use GPT for a single operation (like subset as suggested by @marpet). Move to process one image with a graph containing several steps (e.g.: resample and subset). It is also convenient to understand the structure of the xml graph files. This will help you to solve errors in the future.
Batch processing in GPT is definitely your last move. For that, you need a code (in bash, python, R, or whatever you prefer) that changes the input parameter and run the command. That is why it is not straight forward. You can find this code on the internet but you will need to understand the script to really get what is doing and which are the limitations and possible adaptations to your specific case.
Check the links of @marpet and this material as a starting point:
Don’t be afraid about coding, each hour you spend learning will save days of working time in the future.
@marpet I am trying to batch subset L2 OLCI files (.SEN3 files). When I run this call:
./processDataset.bash subset_4.xml “/media/silsbelab/S31/S3_L2/S3A_20160216_20190724” “/media/silsbelab/S31/S3_L2/Subset_S3_L2” subset_S3_L2
It says “ls: cannot access ‘/media/silsbelab/S31/S3_L2/S3A_20160216_20190724/*.SEN3.nc’: No such file or directory”. Is this not the right call to batch process images?
Update: I got this code to work (I had to change something in the processDataset.bash file)
./processDataset.bash subset_4.xml parameters.properties “/media/silsbelab/S31/S3_L2/S3A_20160216_20190724/” “/media/silsbelab/S31/S3_L2/Subset_S3_L2/” subset_S3_L2
But when it runs, it goes through and says “Error: [NodeId: ReadOp@sourceProduct] Specified ‘file’ [/media/silsbelab/S31/S3_L2/Oa02_reflectance.nc] does not exist.” It seems to be searching for all products listed in OLCI L1 data, how do I change it to L2 products?
I have successfully
resampled a set of S2 imagery according to the example given by Marco Peters to
bulk processing with GPT. Now I am adapting it to apply for a subset process.
I have got some
doubts about the .xml file and Parameters File. I attach them.
The parameters look good.
You can remove the spaces in front of the Polygon parameter. But this shouldn’t be the issue.
That you don’t get a warning or an error and also not processed output is strange.
Is something logged? Maybe there is in the script something wrong so the gpt is not correctly called?
Can you post the complete polygon in textual form?
By the way, in the console output it is stated that you still run SNAP 6.
It might work better with SNAP 7.
Yes.
POLYGON=((-1.7246453762054443 39.6510009765625, -1.2131989002227783 39.6510009765625, -1.2131989002227783 39.1479377746582, -1.7246453762054443 39.1479377746582, -1.7246453762054443 39.6510009765625, -1.7246453762054443 39.6510009765625))
Also, I attach you the three files that I am applying. myGraph_T30_SXJ_resto.xml (645 Bytes) processDataset_resto.bat (2.2 KB) subset.properties (284 Bytes).