that is clearly not enough for a RF classifier. It is based on the idea that the input rasters and pixels are randomly shuffled and selected and only subsets are used for training the dataset. Repeating this, you get a very robust classifier.
It is well explained here: http://wgrass.media.osaka-cu.ac.jp/gisideas10/papers/04aa1f4a8beb619e7fe711c29b7b.pdf
You can include GLCM textures to increase your feature space.