Accurately forecasting power generation in photovoltaic (PV) installations is a challenging task, due to the volatile and highly intermittent nature of solar-based renewable energy sources. In recent years, several PV power generation forecasting models have been proposed in the relevant literature. However, there is no consensus regarding which models perform better in which cases. Moreover, literature lacks of works presenting detailed experimental evaluations of different types of models on the same data and forecasting conditions. This paper attempts to fill in this gap by presenting a comprehensive benchmarking framework for several analytical, data-based and hybrid models for multi-step short-term PV power generation forecasting. All models were evaluated on the same real PV power generation data, gathered from the realisation of a small scale pilot site in Thessaloniki, Greece. The models predicted PV power generation on multiple horizons, namely for 15 min, 30 min, 60 min, 120 min and 180 min ahead of time. Based on the analysis of the experimental results we identify the cases, in which specific models (or types of models) perform better compared to others, and explain the rationale behind those model performances.