Riordan matrices are infinite lower triangular matrices determined by a pair of formal power series over the real or complex field. These matrices have been mainly studied as combinatorial objects with an emphasis placed on the algebraic or combinatorial structure. The present paper contributes to the linear algebraic discussion with an analysis of Riordan matrices by means of the interaction of the properties of formal power series with the linear algebra. Specifically, it is shown that if a Riordan matrix A is an n×n pseudo-involution then the singular values of A must come in reciprocal pairs. Moreover, we give a complete analysis of existence and nonexistence of the eigenvectors of Riordan matrices. This leads to a surprising partition of the group of Riordan matrices into matrices with three different types of sets of eigenvectors. Finally, given a nonzero vector v, we investigate the Riordan matrices A that stabilize the vector v, i.e.Av=v.