We present VLT/SINFONI imaging spectroscopy of the rest-frame optical emission lines of warm ionized gas in 33 powerful radio galaxies at redshifts z ≳ 2, which are excellent sites to study the interplay of rapidly accreting active galactic nuclei and the interstellar medium of the host galaxy in the very late formation stages of massive galaxies. Our targets span two orders of magnitude in radio size (2−400 kpc) and kinetic jet energy (a few 1046– almost 1048 erg s-1). All sources have complex gas kinematics with broad line widths up to ~1300 km s-1. About half have bipolar velocity fields with offsets up to 1500 km s-1 and are consistent with global back-to-back outflows. The others have complex velocity distributions, often with multiple abrupt velocity jumps far from the nucleus of the galaxy, and are not associated with a major merger in any obvious way. We present several empirical constraints that show why gas kinematics and radio jets seem to be physically related in all galaxies of the sample. The kinetic energy in the gas from large scale bulk and local outflow or turbulent motion corresponds to a few 10-3 to 10-2 of the kinetic energy output of the radio jet. In galaxies with radio jet power ≳ 1047 erg s-1, the kinetic energy in global back-to-back outflows dominates the total energy budget of the gas, suggesting that bulk motion of outflowing gas encompasses the global interstellar medium. This might be facilitated by the strong gas turbulence, as suggested by recent analytical work. We compare our findings with recent hydrodynamic simulations, and discuss the potential consequences for the subsequent evolution of massive galaxies at high redshift. Compared with recent models of metal enrichment in high-z AGN hosts, we find that the gas-phase metallicities in our galaxies are lower than in most low-z AGN, but nonetheless solar or even super-solar, suggesting that the ISM we see in these galaxies is very similar to the gas from which massive low-redshift galaxies formed most of their stars. This further highlights that we are seeing these galaxies near the end of their active formation phase.