The glassy state is known to undergo slow structural relaxation, where the system progressively explores lower free-energy minima which are either amorphous (ageing) or crystalline (devitrification). Recently, there is growing interest in the unusual intermittent collective displacements of a large number of particles known as 'avalanches'. However, their structural origin and dynamics are yet to be fully addressed. Here, we study hard-sphere glasses which either crystallize or age depending on the degree of size polydispersity, and show that a small number of particles are thermodynamically driven to rearrange in regions of low density and bond orientational order. This causes a transient loss of mechanical equilibrium which facilitates a large cascade of motion. Combined with previously identified phenomenology, we have a complete kinetic pathway for structural change which is common to both ageing and crystallization. Furthermore, this suggests that transient force balance is what distinguishes glasses from supercooled liquids.