•Combined with the error reconstruction rate, an adaptive VMD technology is developed and used for data preprocessing.•Two mainstream deep learning methods (LSTM, DBN) are used as sub-series predictors, and the classic PSO algorithm is utilized to determine the number of neurons in the DBN network.•A novel hybrid prediction model based on the PSO-DBN nonlinear combination mechanism is proposed. Wind power forecasting plays a vital role in enhancing the efficiency of power grid operation and increasing the competitiveness of power market. In this paper, a novel hybrid forecasting model is developed by using the decomposition strategy, nonlinear weighted combination, and two deep learning models to overcome the drawbacks of the linear weighted combination and further enhance wind power forecasting accuracy and stability. Firstly, the variational mode decomposition (VMD) technique is employed to decompose the original wind power series to extract local features. Then, the long short-term memory neural networks (LSTM) and deep belief networks based on particle swarm optimization (PSO-DBN) are utilized to construct sub-series prediction models. Finally, the multiple sub-series forecasting models are integrated by a nonlinear weighted combination method based on PSO-DBN to construct a hybrid model for short-term wind power forecasting. To verify the performance of the developed forecasting model, wind speed data of 10 min from a wind farm in Shaanxi Dingbian, China are selected as case studies. The results show that the proposed method in this paper is more effective than other existing methods.