Assuming the topo-removal step has worked correctly the remaining phase still contains phase-terms related to atmospheric artefacts, errors in the DEM, and terrain movement. It would be instructive to see the InSAR-phase before the topo-removal step.