During the process of plant growth, such as during the flowering stages and fruit development, the plants need to be provided with the various minerals and nutrients to grow. Nutrient deficiency is the cause of serious diseases in plant growth, affecting crop yield. In this article, we employed artificial neural network models to recognize, classify, and predict the nutritional deficiencies occurring in tomato plants (Solanum lycopersicum L.). To classify and predict the different macronutrient deficiencies in the cropping process, this paper handles the captured images of the macronutrient deficiency. This deficiency during the fruiting and leafing phases of tomato plant are based on a deep convolutional neural network (CNN). A total of 571 images were captured with tomato leaves and fruits containing the crop species at the growth stage. Among all images, 80% (461 captured images) were used for the training dataset and 20% (110 captured images) were applied for the validation dataset. In this study, we provide an analysis of two different model architectures based on convolutional neural network for classifying and predicting the nutrient deficiency symptoms. For instance, Inception-ResNet v2 and Autoencoder with the captured images of tomato plant growth under the greenhouse conditions. Moreover, a major type of statistical structure, namely Ensemble Averaging, was applied with two aforementioned predictive models to increase the accuracy of predictive validation. Three mineral nutrients, i.e., Calcium/Ca2+, Potassium/K+, and Nitrogen/N, are considered for use in evaluating the nutrient status in the development of tomato plant with these models. The aim of this study is to predict the nutrient deficiency accurately in order to increase crop production and prevent the emergence of tomato pathology caused by lack of nutrients. The predictive performance of the three models in this paper are validated, with the accuracy rates of 87.273% and 79.091% for Inception-ResNet v2 and Autoencoder, respectively, and with 91% validity using Ensemble Averaging.