The product of exponential model based robot calibration approach eliminates parameter discontinuity and simplifies coordinate frame setup, but demands extra effort to normalize twist coordinates and differentiate parameter-varying exponential maps. In this paper, we show that such an endeavor can be exempted by respecting the nonlinear geometry of the joint axis configuration space (ACS), the set of all possible axis locations. We analyze the geometry of the ACS models for prismatic and revolute joints, and treat the errors as Adjoint transformations on joint twists. We propose a novel robot kinematic calibration algorithm based on the ACS and Adjoint error model. It is geometrically intuitive, computationally efficient, and can easily handle additional assumptions on joint axes relations. We present a comparative study with simulations and experiments to show that our algorithm outperforms the existing ones in various aspects.