The largest Herschel extragalactic surveys, H-ATLAS and HerMES, have selected a sample of "ultrared" dusty star-forming galaxies (DSFGs) with rising SPIRE flux densities (S500 > S350 > S250; the so-called "500 m risers") as an efficient way for identifying DSFGs at higher redshift (z > 4). In this paper, we present a large Spitzer follow-up program of 300 Herschel ultrared DSFGs. We have obtained high-resolution Atacama Large Millimeter/submillimeter Array, Northern Extended Millimeter Array, and SMA data for 63 of them, which allow us to securely identify the Spitzer/IRAC counterparts and classify them as gravitationally lensed or unlensed. Within the 63 ultrared sources with high-resolution data, ∼65% appear to be unlensed and ∼27% are resolved into multiple components. We focus on analyzing the unlensed sample by directly performing multiwavelength spectral energy distribution modeling to derive their physical properties and compare with the more numerous z ∼ 2 DSFG population. The ultrared sample has a median redshift of 3.3, stellar mass of 3.7 × 1011 M , star formation rate (SFR) of 730 M yr−1, total dust luminosity of 9.0 × 1012 L , dust mass of 2.8 × 109 M , and V-band extinction of 4.0, which are all higher than those of the ALESS DSFGs. Based on the space density, SFR density, and stellar mass density estimates, we conclude that our ultrared sample cannot account for the majority of the star-forming progenitors of the massive, quiescent galaxies found in infrared surveys. Our sample contains the rarer, intrinsically most dusty, luminous, and massive galaxies in the early universe that will help us understand the physical drivers of extreme star formation.