We present infrared observations of 66 starburst galaxies over the full range of oxygen abundances observed in local star-forming galaxies, from image to 8.9. The data include imaging and spectroscopy from the Spitzer Space Telescope, supplemented by ground-based near-infrared imaging. We confirm a strong correlation of aromatic emission with metallicity, with a threshold at image. We show that the far-infrared color temperature of the large dust grains increases toward lower metallicity, peaking at a metallicity of 8 before turning over. We compute dust masses and compare them to H i masses from the literature to derive the ratio of atomic gas to dust, which increases by nearly 3 orders of magnitude between solar metallicity and a metallicity of 8, below which it flattens out. The abrupt change in aromatic emission at mid-infrared wavelengths thus appears to be reflected in the far-infrared properties, indicating that metallicity changes affect the composition of the full range of dust grain sizes that dominate the infrared emission. Although the great majority of galaxies show similar patterns of behavior as described above, there are three exceptions, SBS 0335-052E, Haro 11, and SHOC 391. Their infrared SEDs are dominated energetically by the mid- IR near 24 mum rather than by the 60-200 mum region. In addition, they have very weak near-infrared outputs and their SEDs are dominated by emission by dust at wavelengths as short as 1.8 mum. The latter behavior indicates that the dominant star-forming episodes in them are extremely young. The component of the ISM responsible for the usual far-infrared emission appears to be either missing or inefficiently heated in these three galaxies.