•We identify several pitfalls in the process of extracting and evaluating methods to extract hierarchies in the context of HMC.•We propose using a random hierarchy as a necessary benchmark to ...evaluate the relevance of a good hierarchy, in this way we can establish whether our extracted hierarchy is capturing an existing structure between classes or not.•We demonstrate how the relevance of the hierarchy quality depends on the complexity of the dataset and the complexity of the classifiers used.•We perform experimental evaluation of different approaches for extracting hierarchies to showcase the previous contributions.
Using hierarchies of classes is one of the standard methods to solve multi-class classification problems. In the literature, selecting the right hierarchy is considered to play a key role in improving classification performance. Although different methods have been proposed, there is still a lack of understanding of what makes a hierarchy good and what makes a method to extract hierarchies perform better or worse.
To this effect, we analyze and compare some of the most popular approaches to extracting hierarchies. We identify some common pitfalls that may lead practitioners to make misleading conclusions about their methods. To address some of these problems, we demonstrate that using random hierarchies is an appropriate benchmark to assess how the hierarchy’s quality affects the classification performance.
In particular, we show how the hierarchy’s quality can become irrelevant depending on the experimental setup: when using powerful enough classifiers, the final performance is not affected by the quality of the hierarchy. We also show how comparing the effect of the hierarchies against non-hierarchical approaches might incorrectly indicate their superiority.
Our results confirm that datasets with a high number of classes generally present complex structures in how these classes relate to each other. In these datasets, the right hierarchy can dramatically improve classification performance.
A binary classification problem is common in medical field, and we often use sensitivity, specificity, accuracy, negative and positive predictive values as measures of performance of a binary ...predictor. In computer science, a classifier is usually evaluated with precision (positive predictive value) and recall (sensitivity). As a single summary measure of a classifier's performance,
score, defined as the harmonic mean of precision and recall, is widely used in the context of information retrieval and information extraction evaluation since it possesses favorable characteristics, especially when the prevalence is low. Some statistical methods for inference have been developed for the
score in binary classification problems; however, they have not been extended to the problem of multi-class classification. There are three types of
scores, and statistical properties of these
scores have hardly ever been discussed. We propose methods based on the large sample multivariate central limit theorem for estimating
scores with confidence intervals.
An important subfield of brain–computer interface is the classification of motor imagery (MI) signals where a presumed action, for example, imagining the hands' motions, is mentally simulated. The ...brain dynamics of MI is usually measured by electroencephalography (EEG) due to its noninvasiveness. The next generation of brain–computer interface systems can benefit from the generative deep learning (GDL) models by providing end‐to‐end (e2e) machine learning and increasing their accuracy. In this study, to exploit the e2e‐property of deep learning models, a novel GDL methodology is proposed where only minimal objective‐free preprocessing steps are needed. Furthermore, to deal with the complicated multi‐class MI–EEG signals, an innovative multilevel GDL‐based classifying scheme is proposed. The effectiveness of the proposed model and its robustness against noisy MI–EEG signals is evaluated using two different GDL models, that is, deep belief network and stacked sparse autoencoder in e2e manner. Experimental results demonstrate the effectiveness of the proposed methodology with improved accuracy compared with the widely used filter bank common spatial patterns algorithm.
Binary financial distress prediction (FDP), which categorizes corporate financial status into the two classes of distress and nondistress, cannot provide enough support for effective financial risk ...management. This paper focuses on research on multiclass FDP based on the support vector machine (SVM) integrated with the decomposition and fusion methods. Corporate financial status is subdivided into four states: financial soundness, financial pseudosoundness, moderate financial distress and serious financial distress. Three multiclass FDP models are built by integrating the SVM with three decomposition and fusion methods, i.e., one-versus-one (OVO), one-versus-rest (OVR), and error-correcting output coding (ECOC), and they are, respectively called OVO-SVM, OVR-SVM and ECOC-SVM. Empirical research based on data from Chinese listed companies shows that OVO-SVM overall outperforms OVR-SVM and ECOC-SVM and is preferred for multiclass FDP. In addition, all three models trained on the original highly class-imbalanced training dataset cannot obtain satisfying performance, and the data level preprocessing mechanisms that make class distributions balanced in the training dataset can greatly improve their multiclass FDP performance. Compared with multivariate discriminant analysis (MDA) and multinomial logit (MNLogit), OVO-SVM has significantly higher accuracy for financial pseudosoundness and moderate financial distress and lower accuracy for financial soundness and serious financial distress, resulting in no significant difference among their overall multiclass FDP performance. However, OVO-SVM is still more competitive than MDA and MNLogit in that financial pseudosoundness and moderate financial distress are much more difficult to predict by human expertise than the other two financial states.
Many real-world classification tasks suffer from the class imbalanced problem, in which some classes are highly underrepresented as compared to other classes. In this paper, we focus on multi-class ...imbalance problems which are considerably more difficult to address than two-class imbalanced problems. On this account, we develop a novel and effective procedure, called dynamic ensemble selection for multi-class imbalanced datasets (DES-MI), in which the competence of the candidate classifiers are assessed with weighted instances in the neighborhood. The proposed DES-MI consists of two key components: the generation of balanced training datasets and the selection of appropriate classifiers. To do so, we develop a preprocessing procedure to balance the training dataset which relies on random balance. To select the most appropriate classifiers in the scenario of multi-class imbalance problems, we propose a weighting mechanism to highlight the competence of classifiers that are more powerful in classifying examples in the region of underrepresented competence. We develop a thorough experimental study in order to verify the benefits of DES-MI in handling multi-class imbalanced datasets. The obtained results, supported by the proper statistical analysis, indicate that DES-MI is able to improve the classification performance for multi-class imbalanced datasets.
Thyroid disease is considered one of the most common health disorders, which may lead to various health problems. Recent studies reveal that approximately 42 million people in India face thyroid ...dysfunction or disorder problems. The thyroid hormone is responsible for thyroid disorder which may lead to hypothyroidism or hyperthyroidism problems. TSH (Thyroid Stimulating Hormone), T3 (Triiodothyronine, T3-RIA), FT4 (FT4, Free Thyroxine), T4 (Thyroxine), FTI (Free Thyroxine Index, FTI, T7) are the significant components of thyroid test which is performed to diagnose the behavior of thyroid hormone. However, manual analysis of these parameters on large databases to diagnose and predict hypothyroidism or hyperthyroidism is tedious. In this article, various machine learning-based techniques have been applied to build predictive models, which includes decision tree, random forest, naive Bayes and multiclass classifier and a deep learning (DL) based model Artificial Neural Network (ANN), which is best known for dealing with text data has been applied to predict the class of hypothyroidism. The performance evaluation indicates that the decision tree and random forest provide better results with the highest accuracy of 99.5758% and 99.3107% and very few error rates of 0.0424 and 0.0689, respectively. Furthermore, a comparison among the presented classifiers has been made, and also the proposed model has been compared with previous works, and it has been found that it shows better accuracy as compared to other related works. The DL-based ANN model also offers a competitive accuracy which is 93.8226%. Furthermore, this study can be useful for researchers to identify a suitable model for hypothyroidism detection and classification.
The paper presents an improved-RFC (Random Forest Classifier) approach for multi-class disease classification problem. It consists of a combination of Random Forest machine learning algorithm, an ...attribute evaluator method and an instance filter method. It intends to improve the performance of Random Forest algorithm. The performance results confirm that the proposed improved-RFC approach performs better than Random Forest algorithm with increase in disease classification accuracy up to 97.80% for multi-class groundnut disease dataset. The performance of improved-RFC approach is tested for its efficiency on five benchmark datasets. It shows superior performance on all these datasets.
•Commonly studied scenario considers only binary cancer vs. no cancer classification.•Our system classifies whole slide breast biopsies into five diagnostic categories.•Pipeline of fully ...convolutional networks localizes diagnostically relevant regions.•Convolutional neural network classifies detected regions of interest in whole slides.•Experiments show that our method is compatible with predictions of 45 pathologists.
Generalizability of algorithms for binary cancer vs. no cancer classification is unknown for clinically more significant multi-class scenarios where intermediate categories have different risk factors and treatment strategies. We present a system that classifies whole slide images (WSI) of breast biopsies into five diagnostic categories. First, a saliency detector that uses a pipeline of four fully convolutional networks, trained with samples from records of pathologists’ screenings, performs multi-scale localization of diagnostically relevant regions of interest in WSI. Then, a convolutional network, trained from consensus-derived reference samples, classifies image patches as non-proliferative or proliferative changes, atypical ductal hyperplasia, ductal carcinoma in situ, and invasive carcinoma. Finally, the saliency and classification maps are fused for pixel-wise labeling and slide-level categorization. Experiments using 240 WSI showed that both saliency detector and classifier networks performed better than competing algorithms, and the five-class slide-level accuracy of 55% was not statistically different from the predictions of 45 pathologists. We also present example visualizations of the learned representations for breast cancer diagnosis.
Least square regression is a very popular supervised classification method. However, two main issues greatly limit its performance. The first one is that it only focuses on fitting the input features ...to the corresponding output labels while ignoring the correlations among samples. The second one is that the used label matrix, i.e., zero–one label matrix is inappropriate for classification. To solve these problems and improve the performance, this paper presents a novel method, i.e., inter-class sparsity based discriminative least square regression (ICS_DLSR), for multi-class classification. Different from other methods, the proposed method pursues that the transformed samples have a common sparsity structure in each class. For this goal, an inter-class sparsity constraint is introduced to the least square regression model such that the margins of samples from the same class can be greatly reduced while those of samples from different classes can be enlarged. In addition, an error term with row-sparsity constraint is introduced to relax the strict zero–one label matrix, which allows the method to be more flexible in learning the discriminative transformation matrix. These factors encourage the method to learn a more compact and discriminative transformation for regression and thus has the potential to perform better than other methods. Extensive experimental results show that the proposed method achieves the best performance in comparison with other methods for multi-class classification.
•We perform multi-class classification based on tree tensor network and multiscale entanglement renormalization ansatz.•The agent applies to quantum data by predicting the three phases associated to ...the ground states of a parametrized XXZ model.•Test accuracy turns out to be 96% with the best combination of ansatz and encoding.
Hybrid quantum-classical algorithms based on variational circuits are a promising approach to quantum machine learning problems for near-term devices, but the selection of the variational ansatz is an open issue. Recently, tensor network-inspired circuits have been proposed as a natural choice for such ansatz. Their employment on binary classification tasks provided encouraging results. However, their effectiveness on more difficult tasks is still unknown. In particular, the best approach to extend them to multi-class classification problems remains unclear. Here, we present numerical experiments on multi-class classifiers based on tree tensor network and multiscale entanglement renormalization ansatz circuits. We conducted experiments on image classification with the MNIST dataset and on quantum phase recognition with the XXZ model by Cirq and TensorFlow Quantum. In the former case, we reduced the number of classes to four to match the aimed output based on 2 qubits. The quantum data of the XXZ model consist of three classes of ground states prepared by a checkerboard circuit used for the ansatz of the variational quantum eigensolver, corresponding to three distinct quantum phases. Test accuracy turned out to be 59%-93% and 82%-96% respectively, depending on the model architecture and on the type of preprocessing.
PDF
