An anapole state that breaks inversion and time-reversal symmetries while preserving translation symmetry of an underlying lattice has aroused great interest as a new quantum state, but only a few candidate materials have been reported. Recently, in a spin-orbit coupled Mott insulatorSr2(Ir1−xRhx)O4, the emergence of a possible hidden-order phase with broken inversion symmetry has been suggested atTΩabove the Néel temperature by optical second-harmonic-generation measurements. Moreover, polarized neutron diffraction measurements revealed broken time-reversal symmetry belowTΩ, which was supported by subsequent muon spin relaxation experiments. However, the nature of this mysterious phase remains largely elusive. Here, we investigate the hidden-order phase through the combined measurements of the in-plane magnetic anisotropy with exceptionally high-precision magnetic torque and the nematic susceptibility with elastoresistance. A distinct twofold in-plane magnetic anisotropy along the 110 Ir-O-Ir bond direction sets in below aboutTΩ, providing thermodynamic evidence for a nematic phase transition with brokenC4rotational symmetry. However, in contrast to the even-parity nematic transition reported in other correlated electron systems, the nematic susceptibility exhibits no divergent behavior towardsTΩ. These results provide bulk evidence for an odd-parity order parameter with broken rotational symmetry in the hidden-order state. We discuss the hidden order in terms of an anapole state, in which the polar toroidal moment is induced by two current loops in eachIrO6octahedron of opposite chirality. Contrary to the simplest loop-current pattern previously suggested, the present results are consistent with a pattern in which the intra-unit cell loop current flows along only one of the diagonal directions in theIrO4square.