This work deals with the estimation of terrain topography from multi-pass Synthetic Aperture Radar (SAR) interferometry (InSAR), focusing on the case where the variation of the system geometry within the imaged swath is relevant, as in airborne multi-pass interferometric campaigns. The space varying nature of the system geometry gives rise to a major issue in multi-pass InSAR analyses, in that it prevents from compensating for the presence of interferogram phase offsets by simply phase locking the data stack to a reference point, therefore hindering the retrieval of terrain topography. To cope with this issue properly we propose a novel approach that exploits the algebraic properties of the problem. Such an approach allows to cast the problem in terms of identification of a null space component for terrain topography, after which both topography and the interferogram phase offsets are quickly obtained without exploiting calibration points.