the NaN pixels fall below the confidence threshold of the RF algorithm meaning that the signature cannot be assigned to a class above the desired confidence.
At the moment you can only increase the confidence threshold in the layer properties but nontheless these areas are zero. Would be a nice improvement of the RF module to have more influence on parameters like confidence or number of samples per node ect.