Crop yield predictions are important for crop monitoring and agronomic management. The traditional methods for yield predictions are complicated and resource consuming. With the availability of affordable handheld imaging and computing devices, the image processing-based yield prediction methods are gaining popularity. In this work, RGB images of rice panicles are captured using DSLR camera with simple background and processed to determine the panicle area in terms of number of pixels. A machine learning-based model is developed to make predictions for rice yield. The model is trained to make predictions on unseen data. Various machine learning-based regression algorithms including decision tree, random forest, support vector machine, and convolution neural network are tested. The experiments are performed on a publically available dataset from China as well as on a self-acquired dataset in India. The results have shown that image processing and machine learning-based methods can make yield predictions satisfactorily as evident from the coefficient of determination ( R 2 ) that ranges 0.80–0.97 for different cultivars. The prediction error is determined in terms of root mean square error (RMSE) and mean absolute error (MAE). RMSE for different methods lies between 0.14 and 0.40, whereas MAE varies from 0.11 to 0.30. Among the tested algorithms, linear regression achieved the best precision with R 2 = 0.97, RMSE = 0.14, and MAE = 0.11.