This paper presents a novel asymptotic tracking controller for an underactuated quadrotor unmanned aerial vehicle using the robust integral of the signum of the error (RISE) method and an immersion and invariance (I&I)-based adaptive control methodology. The control system is decoupled into two parts: the inner loop for attitude control and the outer loop for position control. The RISE approach is applied in the inner loop for disturbance rejection, whereas the I&I approach is chosen for the outer loop to compensate for the parametric uncertainties. The asymptotic tracking of the time-varying 3-D position and the yaw motion reference trajectories is proven via the Lyapunov-based stability analysis and LaSalle's invariance theorem. Real-time experiment results, which are performed on a hardware-in-the-loop simulation testbed, are presented to illustrate the performance of the proposed control scheme.