Solar power has rapidly become an increasingly important energy source in many countries over recent years; however, the intermittent nature of photovoltaic (PV) power generation has a significant impact on existing power systems. To reduce this uncertainty and maintain system security, precise solar power forecasting methods are required. This study summarizes and compares various PV power forecasting approaches, including time-series statistical methods, physical methods, ensemble methods, and machine and deep learning methods, the last of which there is a particular focus. In addition, various optimization algorithms for model parameters are summarized, the crucial factors that influence PV power forecasts are investigated, and input selection for PV power generation forecasting models are discussed. Probabilistic forecasting is expected to play a key role in the PV power forecasting required to meet the challenges faced by modern grid systems, and so this study provides a comparative analysis of existing deterministic and probabilistic forecasting models. Additionally, the importance of data processing techniques that enhance forecasting performance are highlighted. In comparison with the extant literature, this paper addresses more of the issues concerning the application of deep and machine learning to PV power forecasting. Based on the survey results, a complete and comprehensive solar power forecasting process must include data processing and feature extraction capabilities, a powerful deep learning structure for training, and a method to evaluate the uncertainty in its predictions.