•Developing more suitable small deep learning model for ore image classification.•Evaluating the gas-coal image classification performance of small deep learning models with different depths.•Optimizing the convergence speed and classification accuracy of small deep learning classification models by adding BN layer.•Evaluating the gas-coal image classification performance of small deep learning models under different dataset sizes. The ore image classification technology based on deep learning is an effective way to improve the image sensor-based ore sorting classification capability. However, in practice, the image sensor-based ore sorting technique often has the problem of insufficient data, and has not systematically considered the impact of model structure and dataset size on the modeling efficiency and classification performance of deep learning. Therefore, this paper attempts to explore a more suitable small deep learning model for ore image classification by considering the model depth, model structure, and dataset size. Six Convolutional Neural Networks (CNNs) models are established with different depths based on Alex Net and VGG Net and the model structure is optimized by adding BN layer. Taking the gas-coal image dataset as case study, we systematically explore the influence of model depth, model structure, dataset size on the training process efficiency and classification accuracy. Meanwhile, the operational process of coal image classifiers is analyzed visually through the ways of Channel Visualization maps, Heatmaps, Grad-CAM map, and Guided Backpropagation maps.