•A federated multi-source domain adaptation method is developed to machinery fault diagnosis with data privacy, which is rarely involved in the existing research.•A federated feature alignment idea ...is introduced to distill common and similar features of all source and target domains.•Two kinds of negative transfer in federated domain adaptation are presented, and a joint voting scheme is designed to achieve the superior results of the target task.
Transfer learning can effectively solve the target task identification problem with the prerequisite of sharing all user data and target data, and has become one of the most popular algorithms in fault diagnosis. However, due to industry competition, privacy security and other factors, transfer learning methods often cannot directly deal with fault diagnosis problems under data privacy. Therefore, a federated multi-source domain adaptation method combining transfer learning and federated learning is proposed for machinery fault diagnosis with data privacy. The proposed method can comprehensively utilize all user data to achieve accurate identification of target data under the premise of data privacy protection. Specifically, a federated feature alignment idea is developed to minimize the difference in feature distribution between different client data and central server data, which can reduce the negative transfer phenomenon in the feature alignment process. Furthermore, a joint voting scheme is designed to fine-tune the global model with the help of pseudo-labeled samples to obtain more accurate fault diagnosis results. Massive experiments suggest that the proposed federated learning method has bright application prospects.
•A novel MDFN is proposed for cross-domain fault diagnosis of rotating machinery.•The domain factorization strategy is elaborated to learn domain-invariant features.•The IET loss term is designed to ...avoid the interference of “bad samples”.•Significant mitigation of negative transfer.
Unsupervised domain adaptation (DA) provides a promising approach for tackling fault diagnosis tasks of target datasets without labeled data and has been actively studied in recent years. Most of them focus only on single-source DA, compared to multisource DA (MDA), which has remarkable advantages in generalized knowledge learning and generalization performance. Nevertheless, there are very few fault diagnosis studies based on MDA, and it remains challenging to reduce multiple domain shifts to improve diagnostic performance and mitigate negative transfer during learning. To this end, a novel unsupervised MDA-based transfer learning approach called multisource domain factorization network (MDFN) is proposed in this paper, where the generalized diagnosis knowledge is learned from multiple sources and then used for diagnosing the target task. The highlights of MDFN are that the shared-space component analysis and transferability-based entropy penalty strategy are employed to significantly mitigate negative transfer from the two levels of feature representation and instance transferability and effectively learn shared feature representation. Therefore, the MDFN can extract shared features that combine domain-invariance and discriminability, thereby performing better. The results of two experimental cases on six datasets, including cross-operating-condition and cross-component diagnosis tasks, validate the effectiveness and superiority of the proposed method.
Universal domain adaptive object detection (UniDAOD) is a more challenging and realistic problem than traditional domain adaptive object detection (DAOD), aiming to transfer the knowledge from the ...well-labeled source domain to the unlabeled target domain without any prior knowledge of label sets. Intuitively, the main challenge of UniDAOD is to eliminate the domain shift and suppress the interference caused by the category shift induced by private classes (i.e., classes only existed in one domain). In the current study, we propose a simple but effective CODE framework, namely Confused and Disentangled Extraction, for alleviating this issue. Specifically, we propose the virtual adversarial adaptation module, characterized by incorporating virtual domain labels within the domain classifier for unaligned samples. This confuses the domain classifier, effectively addressing the issue of converging to local optima resulting from equilibrium challenges and consequently narrowing the domain shift. Simultaneously, we introduce the entropy margin separation module, which utilizes the distinctiveness of category predictions as a disentangled factor. This enables the automatic discovery of private classes in each domain, suppressing interference during the adaptation process. Experiments on four universal scenarios (i.e., closed-set, partial-set, open-partial-set, and open-set) show that CODE obtains a significant performance gain over original DAOD detectors.
In this work, we study the transfer learning problem under high-dimensional generalized linear models (GLMs), which aim to improve the fit on target data by borrowing information from useful source ...data. Given which sources to transfer, we propose a transfer learning algorithm on GLM, and derive its
l
1
/
l
2
-estimation error bounds as well as a bound for a prediction error measure. The theoretical analysis shows that when the target and sources are sufficiently close to each other, these bounds could be improved over those of the classical penalized estimator using only target data under mild conditions. When we don't know which sources to transfer, an algorithm-free transferable source detection approach is introduced to detect informative sources. The detection consistency is proved under the high-dimensional GLM transfer learning setting. We also propose an algorithm to construct confidence intervals of each coefficient component, and the corresponding theories are provided. Extensive simulations and a real-data experiment verify the effectiveness of our algorithms. We implement the proposed GLM transfer learning algorithms in a new R package
glmtrans
, which is available on CRAN.
Supplementary materials
for this article are available online.
Transfer learning is a machine learning paradigm where knowledge from one problem is utilized to solve a new but related problem. While conceivable that knowledge from one task could help solve a ...related task, if not executed properly, transfer learning algorithms can impair the learning performance instead of improving it – commonly known as negative transfer. In this paper, we use a parametric statistical model to study transfer learning from a Bayesian perspective. Specifically, we study three variants of transfer learning problems, instantaneous, online, and time-variant transfer learning. We define an appropriate objective function for each problem and provide either exact expressions or upper bounds on the learning performance using information-theoretic quantities, which allow simple and explicit characterizations when the sample size becomes large. Furthermore, examples show that the derived bounds are accurate even for small sample sizes. The obtained bounds give valuable insights into the effect of prior knowledge on transfer learning, at least with respect to our Bayesian formulation of the transfer learning problem. In particular, we formally characterize the conditions under which negative transfer occurs. Lastly, we devise several (online) transfer learning algorithms that are amenable to practical implementations, some of which do not require the parametric assumption. We demonstrate the effectiveness of our algorithms with real data sets, focusing primarily on when the source and target data have strong similarities.
Effective knowledge transfer has been proven to achieve superior performance in evolutionary optimization. Evolutionary multitasking optimization (EMT), which can solve several optimization tasks ...simultaneously using evolutionary algorithms, continues to be a young research field but is growing rapidly. Due to the parallelism of population-based search, the performance of component tasks can be improved through effective knowledge transfer between different tasks. The main challenge in the EMT field is addressing the negative transfer. Aiming to overcome this challenge, this paper proposes an EMT algorithm that transfers effective knowledge through semi-supervised learning. In addition, a semi-supervised classification method is designed based on the cluster assumption, which is part of the geometric basis of semi-supervised learning. By using both labeled and unlabeled samples generated in the optimization process, the proposed method can identify individuals that contain valuable knowledge and select them to transfer the knowledge between tasks. In this way, the performance of the EMT algorithm can be significantly improved. The effectiveness of the proposed method is verified by empirical tests and comparison with two benchmarks. Further, a case study is conducted. The results indicate that the proposed algorithm can achieve highly competitive performance compared with the state-of-the-art EMT algorithms.
Multi-task learning (MTL) is a joint learning paradigm to improve the generalization performance of the tasks. At present, most of MTL methods are all based on one hypothesis that all learning tasks ...are related and approximate for joint learning. However, this hypothesis may not be held in some scenarios, which may further lead to the problem of negative transfer. Therefore, in this paper, we aim to deal with the negative transfer problem and simultaneously improve the generalization performance in the joint learning. Combining with the subspace learning, we proposed a calibrated multi-task subspace learning method (CMTSL) under the binary group constraint. With the low-rank constraint on subspaces and the binary group indicator, our model can identify “with whom” one task should share and perform the multi-task inference on the high-dimensional parameter space in the meantime. To better approximate the low-rank constraint, we introduce a capped rank function as the tight relaxation term. Last, an iteration based re-weighted algorithm is proposed to solve our model and the convergence analysis is also proved in theory. Experimental results on benchmark datasets demonstrate the superiority of our model.
In the realm of object detection, traditional domain adaptive object detection (DAOD) methods assume that source and target data completely share one identical class space, which is often difficult ...to satisfy in many real-world applications. To address this limitation, this paper introduces universal domain adaptive object detection (UniDAOD), a learning paradigm that relaxes identical class space assumption to be a different but overlapped class space. Intuitively, the main challenge of UniDAOD is to reduce the negative transfer of private classes (i.e., classes only existed in one domain) and reinforce the positive transfer of the common classes (i.e., classes shared across domains). In this paper, we provide a rigorous theoretical analysis and induce a new generalization bound of the expected target error under the UniDAOD setting. On the basis of theoretical insight, we then propose weighted adaptation (W-adapt) to suppress the interference of private classes and reinforce the positive effects of common classes. In particular, we propose a pseudo category margin (PCM) to quantify class importance based on dynamic pseudotarget label prediction to recognize common classes. Furthermore, to alleviate the impact of inaccurate pseudotarget labels, we propose a temporary memory-based filter (TMF) to dynamically store and update the PCM during progressive training. On the basis of the learned TMF, we design a weighted classwise domain alignment loss to adapt two domains across common classes. Experiments on four universal scenarios (i.e., partial-set, open-partial-set, open-set, and closed-set) show that W-adapt outperforms several domain adaptation methods.
In most traditional machine learning algorithms, the training and testing datasets have identical distributions and feature spaces. However, these assumptions have not held in many real applications. ...Although transfer learning methods have been invented to fill this gap, they introduce new challenges as negative transfers (NTs). Most previous research considered NT a significant problem, but they pay less attention to solving it. This study will propose a transductive learning algorithm based on cellular learning automata (CLA) to alleviate the NT issue. Two famous learning automata (LA) entitled estimators are applied as estimator CLA in the proposed algorithms. A couple of new decision criteria called merit and and attitude parameters are introduced to CLA to limit NT. The proposed algorithms are applied to standard LA environments. The experiments show that the proposed algorithm leads to higher accuracy and less NT results.
