We had a simular discussion once (Setting ground control points (GCPs)).
The conclusion was that manual points can never achieve the accuracy of cross-correlation.
Depends a bit on your case.
I think it could be useful if manual GCPs could be used for the determination of the initial offset.