SUMMARY We present an update of the geomagnetic data assimilation tool pygeodyn, use it to analyse ground and satellite-based geomagnetic data sets, and report new findings on the dynamics of the Earth’s outer core on interannual to decadal timescales. Our results support the idea that quasi-geostrophic Magneto-Coriolis waves, recently discovered at a period of 7 yr, also operate on both shorter and longer timescales, specifically in period bands centred around 3.5 and 15 yr. We revisit the source of interannual variations in the length of day and argue that both geostrophic torsional Alfvén waves and quasi-geostrophic Magneto-Coriolis waves can possibly contribute to spectral lines that have been isolated around 8.5 and 6 yr. A significant improvement to our ensemble Kalman filter algorithm comes from accounting for cross-correlations between variables of the state vector forecast, using the ‘Graphical lasso’ method to help stabilize the correlation matrices. This allows us to avoid spurious shrinkage of the model uncertainties while (i) conserving important information contained in off-diagonal elements of the forecast covariance matrix, and (ii) considering a limited number of realizations, thus reducing the computational cost. Our updated scheme also permits us to use observations either in the form of Gauss coefficient data or more directly as ground-based and satellite-based virtual observatory series. It is thanks to these advances that we are able to place global constraints on core dynamics even at short periods.