In agriculture farming business, plant diseases are one of the reasons for the financial deficits around the globe. It is the fundamental factor, as it causes significant abatement in both capacity and quality of the growing crops. In plants, fruits are amongst the major sources of nutrients, however, there exists a wide range of diseases which adversely affect both quality and production of the fruits. To overcome such predicament, computer vision (CV) based methods are introduced. These methods are quite effective, which not only detect the diseases/infections at the early stages but also assign them a label. In this article, we propose a deep convolutional neural network-based method for the diseases classification of different fruits’ leaves. Initially, the deep features are extracted by utilizing pre-trained deep models including VGG-s and AlexNet, which are later fine-tuned by employing a concept of transfer learning. A multi-level fusion methodology is also proposed, prior to the selection step, based on an entropy-controlled threshold value - calculated by averaging the selected features. The resultant final feature vector is later fed into a host classifier, multi-SVM. Five different diseases are considered for experiments including apple black rot, apple scab, apple rust, cherry powdery mildew, and peach bacterial spots, which are collected from a plant village dataset. Classification results clearly reveal the improved performance of proposed method in terms of sensitivity (97.6%), accuracy (97.8%), precision (97.6%), and G-measure (97.6%).