unwrapping depends on clear values without noise which lead from one color into the next along the phase cycle. This figure demonstrates it a bit:
Source: http://vldb.gsi.go.jp/sokuchi/sar/mechanism/mechanism04-e.html
If your image is noisy and doesn’t have this ‘rainbow pattern’ over larger areas (as demonstrated in red), there is no chance that the unwrapped result will reflect the true topography/displacement (blue line).
This topic might help you understand it a bit better: Some explanations about concepts of fringes of interferogram and coherence