We have conducted an archival Spitzer study of 38 early-type galaxies in order to determine the origin of the dust in approximately half of this population. Our sample galaxies generally have good wavelength coverage from 3.6 mu m to 160 mu m, as well as visible-wavelength Hubble Space Telescope (HST) images. We use the Spitzer data to estimate dust masses, or establish upper limits, and find that all of the early-type galaxies with dust lanes in the HST data are detected in all of the Spitzer bands and have dust masses of ~105-106.5 M, while galaxies without dust lanes are not detected at 70 mu m and 160 mu m and typically have <105 M of dust. We propose that dust in early-type galaxies is seeded by external accretion, yet the accreted dust is maintained by continued growth in externally accreted cold gas beyond the nominal lifetime of individual grains. As the majority of dusty early-type galaxies are also low-luminosity active galactic nuclei and likely fueled by this cold gas, their lifetime should similarly be several Gyr.