At present, pointer meters are still widely used because of their mechanical stability and electromagnetic immunity, and it is the main trend to use a computer vision-based automatic reading system to replace inefficient manual inspection. Many correction and recognition algorithms have been proposed for the problems of skew, distortion, and uneven illumination in the field-collected meter images. However, the current algorithms generally suffer from poor robustness, enormous training cost, inadequate compensation correction, and poor reading accuracy. This paper first designs a meter image skew-correction algorithm based on binary mask and improved Mask-RCNN for different types of pointer meters, which achieves high accuracy ellipse fitting and reduces the training cost by transfer learning. Furthermore, the low-light enhancement fusion algorithm based on improved Retinex and Fast Adaptive Bilateral Filtering (RBF) is proposed. Finally, the improved ResNet101 is proposed to extract needle features and perform directional regression to achieve fast and high-accuracy readings. The experimental results show that the proposed system in this paper has higher efficiency and better robustness in the image correction process in a complex environment and higher accuracy in the meter reading process.