A Survey on Transfer Learning Pan, Sinno Jialin; Yang, Qiang
IEEE transactions on knowledge and data engineering,
10/2010, Letnik:
22, Številka:
10
Journal Article
Recenzirano
A major assumption in many machine learning and data mining algorithms is that the training and future data must be in the same feature space and have the same distribution. However, in many ...real-world applications, this assumption may not hold. For example, we sometimes have a classification task in one domain of interest, but we only have sufficient training data in another domain of interest, where the latter data may be in a different feature space or follow a different data distribution. In such cases, knowledge transfer, if done successfully, would greatly improve the performance of learning by avoiding much expensive data-labeling efforts. In recent years, transfer learning has emerged as a new learning framework to address this problem. This survey focuses on categorizing and reviewing the current progress on transfer learning for classification, regression, and clustering problems. In this survey, we discuss the relationship between transfer learning and other related machine learning techniques such as domain adaptation, multitask learning and sample selection bias, as well as covariate shift. We also explore some potential future issues in transfer learning research.
Abstract
As a promising area in machine learning, multi-task learning (MTL) aims to improve the performance of multiple related learning tasks by leveraging useful information among them. In this ...paper, we give an overview of MTL by first giving a definition of MTL. Then several different settings of MTL are introduced, including multi-task supervised learning, multi-task unsupervised learning, multi-task semi-supervised learning, multi-task active learning, multi-task reinforcement learning, multi-task online learning and multi-task multi-view learning. For each setting, representative MTL models are presented. In order to speed up the learning process, parallel and distributed MTL models are introduced. Many areas, including computer vision, bioinformatics, health informatics, speech, natural language processing, web applications and ubiquitous computing, use MTL to improve the performance of the applications involved and some representative works are reviewed. Finally, recent theoretical analyses for MTL are presented.
A large family of algorithms - supervised or unsupervised; stemming from statistics or geometry theory - has been designed to provide different solutions to the problem of dimensionality reduction. ...Despite the different motivations of these algorithms, we present in this paper a general formulation known as graph embedding to unify them within a common framework. In graph embedding, each algorithm can be considered as the direct graph embedding or its linear/kernel/tensor extension of a specific intrinsic graph that describes certain desired statistical or geometric properties of a data set, with constraints from scale normalization or a penalty graph that characterizes a statistical or geometric property that should be avoided. Furthermore, the graph embedding framework can be used as a general platform for developing new dimensionality reduction algorithms. By utilizing this framework as a tool, we propose a new supervised dimensionality reduction algorithm called marginal Fisher analysis in which the intrinsic graph characterizes the intraclass compactness and connects each data point with its neighboring points of the same class, while the penalty graph connects the marginal points and characterizes the interclass separability. We show that MFA effectively overcomes the limitations of the traditional linear discriminant analysis algorithm due to data distribution assumptions and available projection directions. Real face recognition experiments show the superiority of our proposed MFA in comparison to LDA, also for corresponding kernel and tensor extensions
•Different PV module defect detection methods are overviewed.•A joint EL image augmentation approach is proposed and evaluated.•An accurate module defect diagnosis solution is implemented.•The ...proposed defect analysis solution outperforms the existing solutions.
The maintenance of large-scale photovoltaic (PV) power plants is considered as an outstanding challenge for years. This paper presented a deep learning-based defect detection of PV modules using electroluminescence images through addressing two technical challenges: (1) providing a large number of high-quality Electroluminescence (EL) image generation method for the limit of EL image samples; and (2) an efficient model for automatic defect classification with the generated EL image. The EL image generation approach combines traditional image processing technology and GAN characteristics. It can produce a large number of EL image samples with high resolution using a limited number of samples. Then, a convolution neural network (CNN) based model for the automatic classification of defects in an EL image is presented. CNN is used to extract the deep feature of the EL image. It can greatly increase the accuracy and efficiency of PV modules inspection and health management in comparison with the other solutions. The proposed solution is assessed through extensive experiments by using the existing machine learning models, VGG16, ResNet50, Inception V3 and MobileNet, as the comparison benchmarks. The numerical results confirm that the proposed deep learning-based solution can carry out efficient and accurate defect detection automatically using the electroluminescence images.
An increasing number of studies have shown that lncRNAs are crucial for the control of hormones and the regulation of various physiological processes in the human body, and deletion mutations in RNA ...are related to many human diseases. LncRNA- disease association prediction is very useful for understanding pathogenesis, diagnosis, and prevention of diseases, and is helpful for labelling relevant biological information. In this manuscript, we propose a computational model named bidirectional generative adversarial network (BiGAN), which consists of an encoder, a generator, and a discriminator to predict new lncRNA-disease associations. We construct features between lncRNA and disease pairs by utilizing the disease semantic similarity, lncRNA sequence similarity, and Gaussian interaction profile kernel similarities of lncRNAs and diseases. The BiGAN maps the latent features of similarity features to predict unverified association between lncRNAs and diseases. The computational results have proved that the BiGAN performs significantly better than other state-of-the-art approaches in cross-validation. We employed the proposed model to predict candidate lncRNAs for renal cancer and colon cancer. The results are promising. Case studies show that almost 70% of lncRNAs in the top 10 prediction lists are verified by recent biological research. The experimental results indicated that our proposed model had an accurate predictive ability for the association of lncRNA-disease pairs.
A major challenge for collaborative filtering (CF) techniques in recommender systems is the data sparsity that is caused by missing and noisy ratings. This problem is even more serious for CF domains ...where the ratings are expressed numerically, e.g. as 5-star grades. We assume the 5-star ratings are unordered bins instead of ordinal relative preferences. We observe that, while we may lack the information in numerical ratings, we sometimes have additional auxiliary data in the form of binary ratings. This is especially true given that users can easily express themselves with their preferences expressed as likes or dislikes for items. In this paper, we explore how to use these binary auxiliary preference data to help reduce the impact of data sparsity for CF domains expressed in numerical ratings. We solve this problem by transferring the rating knowledge from some auxiliary data source in binary form (that is, likes or dislikes), to a target numerical rating matrix.
In particular, our solution is to model both the numerical ratings and ratings expressed as like or dislike in a principled way. We present a novel framework of Transfer by Collective Factorization (TCF), in which we construct a shared latent space collectively and learn the data-dependent effect separately. A major advantage of the TCF approach over the previous bilinear method of collective matrix factorization is that we are able to capture the data-dependent effect when sharing the data-independent knowledge. This allows us to increase the overall quality of knowledge transfer. We present extensive experimental results to demonstrate the effectiveness of TCF at various sparsity levels, and show improvements of our approach as compared to several state-of-the-art methods.
•A digital twin framework is proposed for the health management of a spacecraft.•The framework could improve the prognosis and decision support capabilities.•Accumulating inspection data could reduce ...prognosis uncertainty.•Information entropy analysis reveals the reasons for better predictions.
Reusable spacecraft can significantly reduce the cost of space travel, while evaluation of the structural health of the craft between flights becomes one of the key issues. A digital twin framework is proposed in this paper to track the life of spacecraft structures. A digital twin is a digital representation of an engineering system. It can simulate, monitor, diagnose, predict states and optimize operations of the real engineering system in real time. The proposed framework can be divided into offline and online stages. It has the functions of diagnosis, model updating, performance evaluation and data storage. To demonstrate the prognosis and decision support capabilities of the framework, a numerical example considering fatigue crack growth in a load-bearing frame is carried out. The method of manufactured solutions is employed for validation. Information entropy and relative entropy are used for measuring the uncertainties in crack length prediction. The results show that through the framework, crack growth model can be updated to have a lower uncertainty. Future crack growth and reusable life can be predicted more accurately using the improved model. With the structural life of the spacecraft quantified by the framework, mission success rates for repeated flights can be maximized at a lower cost.
PDF
