A randomized algorithm for computing a data sparse representation of a given rank-structured matrix A (a.k.a. an H-matrix) is presented. The algorithm draws on the randomized singular value decomposition (RSVD), and operates under the assumption that methods for rapidly applying A and A∗ to vectors are available. The algorithm uses graph coloring algorithms to analyze the hierarchical tree that defines the rank structure to generate a tailored probability distribution from which to draw the random test matrices. The matrix is then applied to the test matrices, and in a final step the matrix itself is reconstructed by the observed input–output pairs. The method presented is an evolution of the “peeling algorithm” of Lin et al. (2011). For the case of uniform trees, the new method substantially reduces the pre-factor of the original peeling algorithm. More significantly, the new technique leads to dramatic acceleration for many non-uniform trees since it constructs test matrices that are optimized for a given tree. The algorithm is particularly effective for kernel matrices involving a set of points restricted to a lower dimensional object than the ambient space, such as a boundary integral equation defined on a surface in three dimensions.