Additive manufacturing (AM) is emerging as an alternative to conventional subtractive manufacturing methods with the goal to deliver unique and complex net or near-net shaped parts. AM components should operate under various loading conditions, from static to complex dynamic multiaxial loadings, therefor, fatigue performance is often a key consideration. Intrinsic AM defects such as Lack of Fusion (LOF) defects, porosities, and un-melted particles are important for fatigue as a local phenomenon which usually starts at stress concentrations. Defects can be minimized by process optimization and/or post-processing but may not be fully eliminated. Full-scale testing, which is typically very costly and often necessary to assess reliability for fatigue performance of safety critical components, could be reduced by robust analytical fatigue performance prediction techniques. This work reviews the literature on the influential microstructural attributes on fatigue performance of AM parts with a focus on generated defects. This includes AM defect characterization and statistical analysis methods, as well as effect of process parameters and post-processing on defects, and consequently fatigue performance. The review also includes defect-based, microstructure-sensitive, and multiscale models proposed in the literature for modeling the effect of defects on fatigue performance and provides an outlook for additional research needed.