You can use the Band Maths to set the areas of no displacement to zero manually, as shown in this tutorial: InSAR Displacement mapping with ERS data
DInSAR is always relative and the results can have offsets, introduced by various error sources.
Especially when larger parts of the image are of low coherence, the analysis of small isolated areas (of high coherence) can be problematic.
To minimize errors, you can try to Filter out low coherence pixels before phase unwrapping
I see no problem with the magnitude of displacement of a single pair here, because it remains the same after unwrapping (one cycle in the interferogram is still one cycle after unwrapping)