Abnormal traffic incidents such as traffic accidents have become a significant health and development threat with the rapid urbanization of many countries. Thus it is critically important to accurately forecast the traffic accident risks of different areas in a city, which has attracted increasing research interest in the research area of urban computing. The challenges of accurate traffic risk forecasting are three-fold. First, traffic accident data in some areas of a city is sparse, especially for a fine-grained prediction, which may cause the zero inflation problem during model training. Second, the spatio-temporal correlations of the traffic accidents occurring in different areas are rather complex and non-linear, which is difficult to capture by existing shallow models like regression. Third, the occurrence of traffic accidents can be significantly affected by various context features including weather, POI and road network features. It is non-trivial to capture the complex associations between the diverse context features and traffic accident risks for building an accurate prediction model. To address the above challenges, this paper proposes a Multi-View Multi-Task Spatio-Temporal Networks (MVMT-STN) model to forecast fine- and coarse-grained traffic accident risks of a city simultaneously. Specifically, to address the data sparsity issue in a fine-grained prediction, we adopt a multi-task learning framework to jointly forecast both fine- and coarse-grained traffic accident risks by considering their spatial associations. For each granularity prediction, we design the channel-wise CNN and multi-view GCN to capture the local geographic dependency and global semantic dependency, respectively. In order to obtain the diverse impacts of the context features on traffic accidents, we also introduce a fusion learning module that integrates the channel-wise and multi-view features learned from different types of the external factors. We conduct extensive experiments over two large real traffic accident datasets. The results show that MVMT-STN improves the performance of traffic accident risk prediction in both fine- and coarse-grained prediction by a large margin compared with existing state-of-the-art methods.