Reliable traffic prediction is critical to improve safety, stability, and efficiency of intelligent transportation systems. However, traffic prediction is a very challenging problem because traffic data are a typical type of spatio-temporal data, which simultaneously shows correlation and heterogeneity both in space and time. Most existing works can only capture the partial properties of traffic data and even assume that the effect of correlation on traffic prediction is globally invariable, resulting in inadequate modeling and unsatisfactory prediction performance. In this paper, we propose a novel end-to-end deep learning model, called ST-3DNet, for traffic raster data prediction. ST-3DNet introduces 3D convolutions to automatically capture the correlations of traffic data in both spatial and temporal dimensions. A novel recalibration (Rc) block is proposed to explicitly quantify the difference of the contributions of the correlations in space. Considering two kinds of temporal properties of traffic data, i.e., local patterns and long-term patterns, ST-3DNet employs two components consisting of 3D convolutions and Rc blocks to, respectively, model the two kinds of patterns and then aggregates them together in a weighted way for the final prediction. The experiments on several real-world traffic datasets, viz., traffic congestion data and crowd flows data, demonstrate that our ST-3DNet outperforms the state-of-the-art baselines.