Classification of GRD product

[ABraun]

Well, one prepares the image and the other its texture. Both are complementary information and important for classification.

[SaiKiran]

Thank You for your valuable suggestions

[SaiKiran]

Hey @ABraun as per your suggestion i have been using GPT. I was able to do the pre-processing work as processing chain in this but so far i could not understand how to extend the code to my own analysis part( ie. to include my own code in that for further processing).

[ABraun]

I don’t think the GPT can directly call code from other software but you can write a routine in GPT which processes the data to a certain point and writes an intermediate output file. Then, this routine calls the external code to process it and then again proceeds with GPT commands.

In windows, this could be done by batch scripting.

[SaiKiran]

How to write a routine in GPT ? Do you have any example code of this kind.

one more thing, Is it easy for a beginner like me to add plugin to my snap desktop using python ?

[ABraun]

It is nicely described here: https://www.computerhope.com/jargon/b/batchfil.htm
You simply list the steps in the correct order, something like this:

gpt -graph1.xml -outfile.tif
external_command.exe outfile1.tif outfile2.tif
gpt graph2.xml outfile2.tif -final.tif

[SaiKiran]

@marpet Running a process through GPT is faster or processing an image by snappy interface is faster ?
and which one is more convenient if I want to do include my own python script in few steps of the processing for my analysis ?

[ABraun]

The performance should be the same but python is known to struggle with the memory allocation and clearing. While GPT fully clears the cache after each processed product in the list (java machine is restarted), python accumulates temporary variables and becomes slow. One reason for this is that python often only uses one processor core to compute (instead of parallel processing) which also makes it potentially slower if you have a strong machine with multiple cores.
A far as I know, there are plans to tackle these problens with the next releases of SNAP, but there is no date for this yet.

To automate the calling of XMLs by the GPT, but with python scripts seems a good trade-off. It is nicely described here: https://senbox.atlassian.net/wiki/spaces/SNAP/pages/70503590/Creating+a+GPF+Graph

You can also write a batch script for this (instead of a python script) as described here:
https://senbox.atlassian.net/wiki/spaces/SNAP/pages/70503475/Bulk+Processing+with+GPT

Also make sure that gpt makes the use of your computational resources (~80% of your RAM is usually suggested). It is described here:

SNAP - Increasing of java heap size in configuration files

https://senbox.atlassian.net/wiki/spaces/SNAP/pages/15269950/SNAP+Configuration

[SaiKiran]

@ABraun 1. Here, in batch files I couldn’t understand how I must give input(which I got as out put from gpt) to the external_command.exe file . I mean in external_command.exe python script how should I write code to take it as input. Kindly, show me with an example external code.
2. Here, we are reading and writing product two times ( 1st in the gpt and 2nd in the external_command ) . Due to this I thought this whole process will take much longer time compared to running one full code with pre-processing and analysis included using snappy. Is this true ?

[ABraun]

hard to tell without knowing what exactly is your external command and what input it takes. Please explain how far you got and at which point you struggle.

Unless all of the data is processed by snappy (and the external command within python) it is probably not avoidable to write the data as an intermediate product before calling the external command.

But this does not necessarily take more time because once an intermediate product is written, the memory (RAM) used to store it is cleared and your machine runs faster for the subsequent steps. Processing too many steps within one chain can be slower as well. So mabye one “breakpoint” is not so bad. And after the process is finished, you can easily delete the intermediate files (also possible from within python).

[SaiKiran]

I am expecting gpt to do preprocessing work(apply orb file - calb - spk filt - terrain flattening- terrain correction- sigmma0 ) to the raw data. I want this finally generated sigmma0 data to be used in my further analysis/processing.
I wrote a python code to carryout my analysis. It include multi segmentation thresholding approach to generate flood inundation map.
I want the whole process to be atomatic ranging from raw data to flood inundation map generation. But here, my whole work is divided in to two parts one is being carried out in gpt(preprocessing) and other part is being done in python(analysis/processing) . Here, i am facing problem in attaching both of them for continous processing .

[ABraun]

You can call the gpt from within python as well

import os

cmd = "gpt preprocessing.xml input_file intermediate_output"
os.system(cmd)

outfile 2 = python.processing(intermediate_output final_output)

[marpet]

As @ABraun suggested you can invoke gpt from python as a separate process, but you can also use the API of the Graph Processing Framework.

Some examples can be found here (by @antonio19812 ):

techforspace/sentinel

Also these forum threads can be helpful:

S1A product Terrain correction frustration

Dear all, I am pretty new in the forum and in the SAR topic and need some help here. I am trying to process the following product: S1A_EW_GRDM_1SDH_20170222T082410_20170222T082510_015394_01942C_945D.SAFE My Regions Of Interest (ROI) are mainly polar regions where there is no DEM available. Usually those ROIs contain only ocean or is a mixture of ocean and land. I would like to perform the following process: 1 Radiometric correction 2 Speckle filter 3 Terrain correction 4 Mosaicking (Plea…

Example script for multiple operations?

I was wondering if anyone has an example of how to write a python snappy script with multiple sequential operators? There’s also a strong chance I’m misunderstanding exactly how to work with snappy. I’ve tried examining a few examples such as NDVI_OP (which I don’t understand how to use) and snappy_flh (which I think I understand but am not sure what multispectral sensor input is required). I’ve also looked at @abgbaumann’s snappy_glcm , and it seems like I could repeat/modify that (func_name & …

Handling sentinel data in python ( working with snappy)

I have recently installed SNAP in my Windows8.1 with 8GB RAM and as I said in my first message I am trying to work with SNAP in python using Snappy. I managed to use it properly with the test dataset that is provided with the snappy module in the SNAP directory. I usually work with numpy to handle and process images or multidimensional arrays. But the problem is that the test dataset has really small size if compared with the usual sizes of a Sentinel dataset. And when I try to read for exam…

[SaiKiran]

In “2. 1. original --> callibrated -> glcm --> terrain correction” can I include terrain flattening after glcm and before terrain correction to improve the accuracy of the classification.

[SaiKiran]

1. When classifying GRD product using algorithms like maximum likelihood or random forest classifier. Is it necessary to convert the data in to decibels after terrain correction? How does it effect the results ?
2. Out of sigmma0 and gamma0 product which suits better for generating binary image of water and non-water using Maxinum likelihood, random forest classifier and thresholding technique ?

[ABraun]

0. you can only calculate terrain flattening if you calibrated to beta0.

1. maximum likelihood is not suitable for radar data because the data is not distributed equally (vh is generally lower than vv), random forest works better. For both cases, conversion to dB is helpful because it changes the histogram [please see the examples here and here]. If you want to include image textures, random forest is necessary because it is based on thresholds and not of clusters. If you don’t want to include textures, please have a look at these comments on feature space here.

2. Please have a look at the difference between Sigma0 and Gamma0 here. As these refer to topographically induced radiometric distortions (and water bodies are usually flat), it barely makes a difference. Calculate a scatter plot between Sigma0 and Gamma0 to see where they are different.

[SaiKiran]

you can only calculate terrain flattening if you calibrated to beta0.

I am aware of it, but I want know will the accuracy improve if the terrain flattening is done on GLCM products before classification.

[ABraun]

Gamma0 is more likely to represent the actual backscatter of a surface (impact of topography is reduced), so basically yes. But for water bodies, the difference will be small.

[SaiKiran]

Hello @ABraun !

import os
cmd = "gpt preprocessing.xml input_file intermediate_output"
os.system(cmd)`

this works perfectly for me . Thank you.
Now, the issue is I am using internet with proxy settings. So, my command prompt is not able to get the internet access. From internet I came to know that we have to give this command to cmd prompt to access internet :

set http_proxy=http://username:password@your_proxy:your_port

When I enter this command in cmd it works fine but i want to pass this from the python code itself as we did for the gpt. How can I achieve this ?

[ABraun]

I don’t know because haven’t done this so far, sorry.
Maybe someone other can help.