•A lightweight tomato target detection algorithm based on YOLOv5 is proposed.•The improved model has a significantly higher detection speed on the CPU platform.•The model is deployed on smartphones using model quantization and an APP is developed to achieve local real-time detection of tomatoes.•The 16-bit quantized model performs the best and the detection speed meets the needs of real-time. The current deep-learning-based tomato target detection algorithm has many parameters; it has drawbacks of large computation, long time consumption, and reliance on high-computing-power devices such as graphics processing units (GPU). In this study, we propose a lightweight improved YOLOv5 (You Only Look Once) based algorithm to achieve real-time localization and ripeness detection of tomato fruits. Initially, this algorithm used a down-sampling convolutional layer instead of the original focus layer, reconstructing the backbone network of YOLOv5 using the bneck module of MobileNetV3. Then, it performs channel pruning for the neck layer to further reduce the model size and uses a genetic algorithm for hyperparameter optimization to improve detection accuracy. We evaluate the improved algorithm using a homemade tomato dataset. The experimental results demonstrated that the improved model number of parameters and floating point operations per second (FLOPs) were compressed by 78% and 84.15% compared to the original YOLOv5s, while the mAP reached 0.969. Meanwhile, the detection speed on the central processing unit (CPU) platform was 42.5 ms, which was 64.88% better. This study further utilized the Nihui convolutional neural network (NCNN) framework to quantize the improved model and developed an Android-based real-time tomato monitoring application (app). Experimental results demonstrated that the 16-bit quantized model achieved an average detection frame rate of 26.5 frames per second (fps) on the mobile side with lower arithmetic power, which was 268% better than the original YOLOv5s, and the model size was reduced by 51.1% while achieving a 93% true detection rate.