Traditional image aesthetic evaluation method usually involves the extraction of a set of relevant image aesthetic features and classification by a classifier trained on the set of features. The system's performance greatly depends on the effectiveness of the features. However, most of these features are carefully hand-crafted for specific datasets and assumed strong prior knowledge. Therefore, these features would not be optimal for general image aesthetic evaluation. The deep convolution neural network (DCNN) has the ability to automatically learn aesthetic features, and network structure of different complexity can learn aesthetic features at different scales and different point of views. Moreover, traditional image features, such as edge and saliency map, can be used as auxiliary information for the DCNN. Therefore, a Network-Paralleled and Data-Paralleled DCNN (NP-DP-DCNN) structure is proposed. The Network-Paralleled DCNN fuses networks of different complexity and the Data-Paralleled DCNN fuses original image data and derived feature maps to learn the aesthetic features from different scales and different point of views. Experimental results show that the proposed NP-DP-DCNN structure is able to achieve better classification performance than many existing methods.