Using Chandra, XMM-Newton, and optical photometric catalogs we study the young X-ray binary (XRB) populations of the Small Magellanic Cloud. We find that the Be/X-ray binaries (Be-XRBs) are observed in regions with star formation rate bursts {approx}25-60 Myr ago. The similarity of this age with the age of maximum occurrence of the Be phenomenon ({approx}40 Myr) indicates that the presence of a circumstellar decretion disk plays a significant role in the number of observed XRBs in the 10-100 Myr age range. We also find that regions with strong but more recent star formation (e.g., the Wing) are deficient in Be-XRBs. By correlating the number of observed Be-XRBs with the formation rate of their parent populations, we measure a Be-XRB production rate of {approx}1 system per 3 x 10{sup -3} M{sub sun} yr{sup -1}. Finally, we use the strong localization of the Be-XRB systems in order to set limits on the kicks imparted on the neutron star during the supernova explosion.