Rotational parameter estimation is fundamental to the focusing of imaging algorithm and inverse synthetic aperture radar (ISAR) image size calibration. Previous research has indicated that scattering points (SPs) in different imaging time windows locate at various associated positions in range-Doppler (RD) planes. This paper exploits the relation to estimate the rotation parameters (RPs) and translation error parameters for medium processing time length and nonuniform rotation situation. The iterative closet point (ICP) method inspires an alternating iterative algorithm consisting of the SP assignment and parameter optimization. In the parameter optimization step, multiple SP set pairs are utilized to construct the objective function that represents the degree of mismatch between point set pairs. A compatible fast SP extracting method is also presented to reduce the computational complexity. The experimental results show that the proposed algorithm can provide fast, accurate, and robust estimation for parameters in nonuniform rotation case.