In the study on sports image classification, the characteristics of human pose increasingly raise concerns of researchers. However, the same posture for human may be resulted from different scenes and scene objects that express diverse action states and meanings. Thus, combination of human pose and event scenes shall be considered so as to improve performance of sports image classification. In recent years, spatial pyramid matching (SPM) has attracted more and more attentions on the field of natural scene categories. Moreover, the high accuracy in image retrieval and image classification has been shown in multiple works. However, SPM can merely consider the absolute locations of the visual words in images. Hence, this paper attempts to take spatial pyramid matching as the basic idea, and combines with Visual Words Spatial Dependence Matrices that describes the relative spatial information. As shown in the experimental results, classification accuracy of the proposed method is improved by approximately 19% compared with SPM, and superior to some other improved SPM methods in the sports image classification.