Recent studies have found a significant evolution and scatter in the relationship between the UV spectral slope (βUV) and the infrared excess (IRX; LIR/LUV) at z > 4, suggesting different dust properties of these galaxies. The total far-infrared (FIR) luminosity is key for this analysis, but it is poorly constrained in normal (main-sequence) star-forming z > 5 galaxies, where often only one single FIR point is available. To better inform estimates of the FIR luminosity, we construct a sample of local galaxies and three low-redshift analogues of z > 5 systems. The trends in this sample suggest that normal high-redshift galaxies have a warmer infrared (IR) spectral energy distribution (SED) compared to average z < 4 galaxies that are used as priors in these studies. The blueshifted peak and mid-IR excess emission could be explained by a combination of a larger fraction of metal-poor interstellar medium being optically thin to ultraviolet (UV) light and a stronger UV radiation field due to high star formation densities. Assuming a maximally warm IR SED suggests a 0.6 dex increase in total FIR luminosities, which removes some tension between the dust attenuation models and observations of the IRX−β relation at z > 5. Despite this, some galaxies still fall below the minimum IRX−β relation derived with standard dust cloud models. We propose that radiation pressure in these highly star-forming galaxies causes a spatial offset between dust clouds and young star-forming regions within the lifetime of O/B stars. These offsets change the radiation balance and create viewing-angle effects that can change UV colors at fixed IRX. We provide a modified model that can explain the location of these galaxies on the IRX−β diagram.