This article proposes a Rao-Blackwellized particle filter (RBPF) for fully mixing state-space models that replace the Kalman filter within the RBPF method with a noise-adaptive Kalman filter. This extension aims to deal with unknown time-varying measurement variances. Consequently, a variational Bayesian (VB) adaptive Kalman filter estimates the conditionally linear states and the measurement noise variances whereas the nonlinear (or latent) states are handled by sequential Monte Carlo (MC) sampling. Thus, by modifying the underlying mathematical framework of RBPF, we construct the Monte Carlo Variational Bayesian (MCVB) filter. A stopping criterion for VB approximations is proposed by employing Tikhonov regularization. Additionally, an analysis of the numerical stability of the proposed filtering mechanism is presented. The performance of the MCVB filter is illustrated in simulations and mobile robot tracking experiments in the presence of measurement model uncertainties.