The improper ferroelectricity inYMnO3and other related multiferroic hexagonal manganites is known to cause topologically protected ferroelectric domains that give rise to rich and diverse physical phenomena. The local structure and structural coherence across the ferroelectric transition, however, were previously not well understood. Here, we reveal the evolution of the local structure with temperature inYMnO3using neutron total scattering techniques, and we interpret them with the help of first-principles calculations and with a first-principles-based effective Hamiltonian. The results show that, at room temperature, the local and average structures are consistent with the established ferroelectricP63cmsymmetry. On heating, both local and average structural analyses show striking anomalies from about 800 K up to the Curie temperature and signatures of a locally more preserved structure than on average, consistent with increasing fluctuations of the order-parameter angle. These fluctuations result in an unusual local symmetry lowering into a continuum of structures on heating. This local symmetry breaking persists into the high-symmetry nonpolar phase, constituting an unconventional type of order-disorder transition, and we pinpoint it as the reason for the anomalous behavior near the phase transition. The hidden disorder revealed inYMnO3by total scattering is expected to find analogies in other materials with structural frustration or characteristic energy barriers of different magnitudes.