Geocode matrix - external DEM error

atoco · March 25, 2020, 7:45pm

I have T3 matrix which I would like to geocode based on an external DEM (WGS84 datum). However, I get an error:

can’t read “TMPConfigHdr”: no such variable
can’t read “TMPConfigHdr”: no such variable
while executing
“TextEditorRunTrace “Arguments: -id \x22$SNAPDirOutput\x22 -if \x22$TMPConfigHdr\x22 -ss $SNAPSensor -pp $PolarType” “k””
invoked from within
"if [file exists $TMPConfigTxt] {
DeleteFile $SNAPParameterFileInput
DeleteFile “$SNAPParameterFileInput.hdr”
DeleteFile “$SNAP…”
invoked from within
“if [file exists $SnapReader] {
TextEditorRunTrace “Process The Function SNAP/gpt” “k”
TextEditorRunTrace “Arguments: \x22$TMPSNAPBatchCfg\x22”…”
(procedure “SNAPBatchProcess” line 47)
invoked from within
“SNAPBatchProcess $SNAPParameterFile $SNAPParameterOutput”
(procedure “SNAPMatrix_T3” line 11)
invoked from within
"SNAPMatrix_T3 "
invoked from within
“.top427.fra59.but93 invoke”
(“uplevel” body line 1)
invoked from within
“uplevel #0 [list $w invoke]”
(procedure “tk::ButtonUp” line 24)
invoked from within
“tk::ButtonUp .top427.fra59.but93”
(command bound to event)

SAR input image is fully-polarimetric RADARSAT-2, which was only multilooked prior to the Geocoding.
The DEM has to be external…

If there’s no simple solution to this bug in PolSARPro, could you please tell me how to export the T3 Matrix to SNAP format, geocode it directly in SNAP and then return to the PolSARPro?

I would really appricieate any help, I’m dealing with this issue for a loooong time and deadlines are approaching quicly…

ABraun · March 26, 2020, 9:28am

SNAP (called by PolSARpro) needs external DEMs stored as GeoTiffs. Is this the case?
Please also check if the standard SRTM 1Sec AutoDownload works for a first try.

atoco · March 26, 2020, 9:49am

@ABraun, thank you for your quick answer.

My AOI (the Arctic) is out of SRTM 1sec coverage area so I can’t check it (that’s why it has to be external DEM).

The DEM which I’m using is GeoTIFF, WGS84 system, NoData values written as 0 - so I do hope everything is as PolSARPro/SNAP requires (or maybe have I missed something?..).

The Range-Doppler Terrain Correction in SNAP, based on the same input RADARSAT-2 and DEM dataset, works well…

ABraun · March 26, 2020, 9:52am

I see… Strange that the RD Terrain Correction works from within SNAP but not when called by PolSAR pro.
Is it an option to test the ACE30 instead or GETASSE?
Maybe you can check in SNAP if the geolocation is different for the different DEM inputs.

If nothing of them works, you can also compute the T-matrix in SNAP: Interferometry Tutorial with Radarsat-2 (DEM generation)

atoco · March 29, 2020, 9:46pm

I have already tested the ASTER option in PolSARPro (there’s S.T.R.M., ASTER, External). Only with ASTER there is no error however the result doesn’t look Terrain Corrected (but in WGS84 datum). Why this conclusion? Because I have compared the results of TC from PolSARPro and SNAP.

I’ve also tried to migrate data between PSP and SNAP with poor results…
So, what i’m doing now is:
[1] I perform Multilooking -> generation T3 matrix -> Terrain Correction in SNAP. I export the results to PolSARPro format.
[2] In PolSARPro I do the standard operation of extracting T3 matrix with the same multilooking parameters as in SNAP. Then I do the “false” Terrain Correction with ASTER.
[3] Then I replace the files in T3 folder in PolSARPro TC results, with the files from SNAP (i.e. [1] step).
BUT!
The output from PolSARPro and SNAP differs in the number of rows and columns in config.txt file (by 1 and 3 respectively). SNAP does not generate config_mapinfo.txt file.
When I perform e.g. H-alpha Wishart classification in PolSARPro based on the config.txt from SNAP and config_mapinfo.txt from PolSARPro (as there is no such file from SNAP) the result has “stripes”.
When config_mapinfo.txt is deleted then the result has no stripes.
SO…
Ideally, I would like to edit the config_mapinfo.txt generated by PolSARPro based on geolocation from SNAP. But I do not understand the whole the config_mapinfo.txt structure…
Therefore, I would be very, very grateful for the explanation “what is what” in the text below. I do hope this will resolve all my problems… I
BTW This config_mapinfo.txt comes from PolSARPro tutorials, not my data, so the file should be correct:

Sensor
ALOS2
---------
MapInfo
map info = {Geographic Lat/Lon,1647.0,1837.5,-122.38085443768024,37.94843536247002,1.8437278049532324E-4,1.8437278049532324E-4,WGS84,units=Degrees}
---------
MapProj
Geographic Lat/Lon
1647.000000
1837.500000
-122.380854
37.948435
1.843728e-004
1.843728e-004
WGS-84
units=Degrees

Thanks for your help!

ABraun · March 30, 2020, 8:35am

sorry if I missed your point, but why don’t you generate the T-matrix and terrain correct it entirely in SNAP?

atoco · March 30, 2020, 8:54am

Because starting from T3 matrix I can apply many polarimetric analysis in PolSARPro. Building a matrix (not necessarily T3. It can be also T4, C3 or C4) is a first step in this software (= extract data step) so without the matrix I cannot e.g. perform Decomposition or H-alpha Wishart classification.

So for example, for H-alpha Wishart classification the recommended processing chain is as follows:
1. (Multilook &) T3 matrix
2. Geocode matrix (= Terrain Correction) - failed in PSP, I want to replace the files with SNAP results
3. H/A/alpha decomposition
4. Wishart H/A/alpha Classification
(this is taken from tutorials of PSP - please, correct me if something’s wrong)

I know SNAP has many polarimetric tools (including mentioned H-alpha Wishart) but I would like to proceed with PSP and play with other features, not accessible in SNAP…

ABraun · March 30, 2020, 9:44am

I understand, that is legit.

But I don’t have a solution why PolSARpro does not like your DEM. Maybe it is stored in a path with special characters?

atoco · March 30, 2020, 11:21am

No, I’ve just checked with super-simple and short path, the same error as in my first post occurs…

I’m thinking, is there any requirement regarding data format? I mean Float or Double etc.?..

ABraun · March 30, 2020, 11:29am

If I remember correctly, both Integer and Float are accepted but it would be worth a try.
How about this: Conduct the Terrain Correction in SNAP and select “DEM” as additional output band. Then convert this elevation to GeoTiff and hand it over to the Terrain Correction operator in PolSAR pro.

atoco · March 31, 2020, 9:02pm

Good idea, I haven’t thought about it!

But… doesn’t work as well. I’ve checked the output DEM from SNAP in ArcMap, as well as “DEM” from ASTER PolSARPro.
Both Float point, ASTER has -9999 for NoData and it is written in .bin format.
So, I’ve tested my DEM (digested by SNAP TC prior to the PSP):

NoData=-9999; .bin format
NoData=0; .bin format
NoData=-9999; .tif format
NoData=0; .tif format
Nothing above works in PSP…

I don’t know if this is relevant but in the PSP Work Trace window the error from my first post occurs every time when Process The Function Soft/bin/tools/SNAP_mapinfo_config_file.exe is started…

ABraun · April 1, 2020, 7:18am

hmm, is Ellipsoid Correction an option? I’m not sure if it is available from PolSARpro, but if your area is flat, it could be a solution at least.

atoco · April 5, 2020, 3:37pm

unfortunately, the terrain differences are quite high (in some areas with shadowing effect etc.)…

I’m testing the idea of replacing products of geocoding in PolSARPro by SNAP results. Based on metadata I built config_mapinfo.txt and other files which are absent in SNAP results.
The only thing I’m missing now is mask_valid_pixels.bin file. Without this, the results of further processing in PolSARPro are corrupted.

PolSARPro, after “geocoding”, produces four files:
mask_valid_pixels.bin
mask_valid_pixels.bin.hdr
mask_valid_pixels.bmp
mask_valid_pixels.bmp.hdr
Those files are missing in files of T3 matrix geocoded in SNAP and exported to PolSARPro format.

So, I’m trying to build those files myself. At this point, I’m not sure about the .bmp and .bmp.hdr files (if I created them correctly). I am sure however, I have a problem with mask_valid_pixels.bin.
What is my processing chain? I’m changing values of all valid pixels of e.g. T11 band to “1” by Band Maths. Then, in properties I uncheck “NoData values used” and change their values to “0”. Finally, I export the file to Generic Binary BSQ format. As a result, I receive:
mask_valid_pixels.bin
mask_valid_pixels.xml
There is a lack of mask_valid_pixels.bin.hdr file (in comparison to PSP). This file is a text file and stores information on e.g. the number of rows, columns or coordinate system. So I created the file myself. However, on the contrary to the files exported from PolSARPro, those files are not readable by ArcMAP.
I’ve also tried to import a T11 band exported to .bin format from SNAP (so without my band maths etc.) to ArcMAP, with no success.
Therefore, I assume the mask_valid_pixels.bin file from SNAP is somehow corrupted…

I would be very grateful for help on how to export SNAP product to .bin format to be readable by PolSARPro (and probably also ArcMAP)…
Or is it possible to build a mask of valid pixels in PolSARPro based on T3 matrix and .kml/other files?

atoco · April 5, 2020, 4:23pm

SOLVED.

Instead using SNAP to generate pixel mask I used ArcMap.

SNAP: T11 (or another “band”) export to GeoTIFF.
ArcMap: import of the GeoTIFF file. Raster calculator: replacing valid pixels by “1”.
Export to .flt.
Change of the .flt format to .bin “manually” in File Explorer.
mask_valid_pixels.bin.hdr built based on metadata from SNAP, following scheme of PolSARPro file.
mask_valid_pixels.bin and mask_valid_pixels.bin.hdr pasted to T3 folder of PolSARPro (where already other T3 files from SNAP were pasted).

@ABraun, thanks a lot for your help in this topic!

ABraun · April 6, 2020, 10:49am

thank you very much for sharing this workaround!

So it was a matter of the valid pixel mask used in PolSARpro?
How did you find out?

atoco · April 7, 2020, 1:41pm

In my case, the problem during Geocode Matrix in PSP is hidden in Soft/bin/tools/SNAP_mapinfo_config_file.exe function. I don’t know what’s behind this function and what is the problem. This observation is based on PSP Work Trace window.

Therefore, I geocoded T3 matrix in SNAP. I faced the problem of replacing the data from SNAP in PSP. Here, except building e.g. config_mapinfo.txt file (based on metadata), the main problem was “how to build mask_valid_pixels.bin file?”. I found this out based on my many, many attempts to overcome this isssue.