One of the major design goals of modern communication systems is to make the system green, i.e., power efficient. Since power amplifiers (PAs) occupy a major portion of the communication system’s power expenditure, it is critical to enhance the power efficiency of PAs. There are various approaches to optimizing PA efficiencies, like using PAs from more efficient PA classes such as the Doherty PA to achieve design requirements for lower power consumption. Unfortunately, making the PA more efficient usually comes at a cost of higher nonlinear distortion (NLD) that degrades communication receiver performance. The NLD problem has become more challenging today as communication systems continue to adopt denser quadrature amplitude modulation (QAM) constellations, like 4,096-QAM, as well as orthogonal frequency division multiplexing (OFDM) signaling techniques that are extremely sensitive to NLD. The impact NLD imposes on communication systems can be seen across multiple industries. Effective techniques are highly needed to mitigate the effects of NLD.This dissertation gives a systematic study of the PA NLD problem with a special focus on the application of artificial intelligence (AI), or machine learning (ML), to model NLD and to mitigate NLD. After reviewing existing NLD literature and practices, this dissertation showed that almost all the existing works addresses mild to moderate NLD only, not severe NLD. The review also showed the advantage of neural networks to deal with the NLD problem. As severe NLD will play a significant role in future power-efficient system design, this dissertation develops novel strategies that adopt deep neural networks to mitigate severe NLD.In Chapter 1, the application of PA and the associated NLD are introduced in cable TV communication system applications. In Chapter 2, since many existing works have shown that AI techniques consistently outperform traditional techniques in mitigating NLD, we provide a brief introduction to AI techniques and some ML models of NLD. We focus on specifically the Bayesian Network and Neural Network (NN) models.In Chapters 3 through 5, some NLD mitigation practices adopted in today’s practical systems are described. These chapters show that existing practices can mitigate mild to moderate NLD only, not severe NLD. Chapter 3 provides several conventional mathematical/analytical models of NLD. Digital pre-distortion (DPD) is discussed in Chapter 4. Post-distortion is then discussed in Chapter 5.In Chapter 6, we focus on severe NLD and develop a deep learning strategy to mitigating severe NLD at the receiver. Based on the Volterra model, deep neural networks (DNNs) are developed to learn the best NLD equalizer. A critical advantage of this strategy hinges on the capability of equalizing severe NLD, whereas conventional approaches were not competitive. We demonstrate that our proposed method outperforms traditional methods like the Volterra equalizer or neural network-based equalizer with both simulated data and real experiment data. This fact is both enlightening and motivating to further investigate the deep learning techniques for severe NLD equalization in the future.In the concluding Chapter 7, all the concepts of this dissertation, e.g., NLD, AI, Transmitter-side DPD and Receiver-side equalization, are proposed to integrate into one cohesive story that will lead to a robust NLD mitigation strategy that enables more efficient communications systems design and a better understanding of the application of deep learning.