User and producer accuracies with nan

Microwave Toolbox
1

Hi. Has anyone experienced UA/PA with nan in some categories? I know that nan means ‘not a number’ but in this context I dont understand why my accuracies have this.

Thanks

Pamela

2

If your training samples are too small, for example, calculation of accuracies can not be conducted. This could be the reason, especially because only some of your classes are affected.

3

Thanks! The features for that class were very few and scattered within the image, probably that is why. What should I do then?

Also, in another classification, I notice that I have 96% OA with 91% Kappa. Yet 2 classes out of 9 have 0 UA and PA. How does that happen?

Regards,

Pamela

4

the accuracies just show you how good your classifier performed within the training areas. So if there is nothing in your rasters, that can be used to separate a certain class from others, it cannot be modeled by the classifier. The training accuracy, given by SNAP, tells you which classes can be recognized by the classifier and which ones have no distinct signature.

5

So is it ok to report ‘nan’ in the UA/PA?

6

of course not :slight_smile:

7

What would you recommend as a way forward in this case? redo the analysis?

8

I would usggest to use larger training areas which don’t contain invalid pixels.

As an alternative you can perform the accuracy assessment manually, for example in QGIS: http://web.pdx.edu/~nauna/resources/9-accuracyassessment.pdf

But, again, this is only the training accuracy which doesn’t tell you anything about the accuracy of the classification. Please also see here: GLCM worsening accuracy results?

9

Thanks. Unfortunately the sizes of the classes with nan or zero accuracies are very very small compared to the rest of the landscape.

The accuracy assessment that I performed was in QGIS using a 70-30 partitioning.

10

I see, sorry for the misunderstanding. Some people mix those two up…
Is there any chance to include larger parts for those small validation samples?

11

Thanks. Its a bit difficult since that would mean literally selecting every feature from that class in the image.

Let me see whether I can increase the sample sizes.