•Evaluation of LSTM deep neural networks for irradiance forecasting.•New experimental framework including virtual PV site construction.•Approximation of GHI readings using satellite images.•Large-scale benchmark of forecasting methods across 21 geo-locations. Accurate forecasts of solar energy are important for photovoltaic (PV) based energy plants to facilitate an early participation in energy auction markets and efficient resource planning. The study concentrates on Long Short Term Memory (LSTM), a novel forecasting method from the family of deep neural networks, and compares its forecasting accuracy to alternative methods with a proven track record in solar energy forecasting. To provide a comprehensive and reliable assessment of LSTM, the study employs remote-sensing data for testing predictive accuracy at 21 locations, 16 of which are in mainland Europe and 5 in the US. To that end, a novel framework to conduct empirical forecasting comparisons is introduced, which includes the generation of virtual PV plants. The framework enables richer comparisons with higher coverage of geographical regions. Empirical results suggest that LSTM outperforms a large number of alternative methods with substantial margin and an average forecast skill of 52.2% over the persistence model. An implication for energy management practice is that LSTM is a promising technique, which deserves a place in forecasters’ toolbox. From an academic point of view, LSTM and the proposed framework for experimental design provide a valuable environment for future studies that assess new forecasting technology.