this happens when there is much dihedral scattering. Lower the Adaptive Filter Exponent to 0.3 or lower as suggested in this tutorial DEM generation with Sentinel-1 - Workflow and challenges (page 26)
Same was discussed here: Goldstein filtering introducing artifacts around bright targets