In this paper, we prove several new Hardy type inequalities (such as the weighted Hardy inequality, weighted Rellich inequality, critical Hardy inequality and critical Rellich inequality) related to the radial derivation (i.e., the derivation along the geodesic curves) on the Cartan–Hadamard manifolds. By Gauss lemma, our new Hardy inequalities are stronger than the classical ones. We also establish the improvements of these inequalities in terms of sectional curvature of the underlying manifolds which illustrate the effect of curvature to these inequalities. Furthermore, we obtain some improvements of Hardy and Rellich inequalities on the hyperbolic space ℍn. Especially, we show that our new Rellich inequalities are indeed stronger than the classical ones on the hyperbolic space ℍn.