Multimodal Federated Learning: A Survey Che, Liwei; Wang, Jiaqi; Zhou, Yao ...
Sensors (Basel, Switzerland),
08/2023, Letnik:
23, Številka:
15
Journal Article
Recenzirano
Odprti dostop
Federated learning (FL), which provides a collaborative training scheme for distributed data sources with privacy concerns, has become a burgeoning and attractive research area. Most existing FL ...studies focus on taking unimodal data, such as image and text, as the model input and resolving the heterogeneity challenge, i.e., the challenge of non-identical distribution (non-IID) caused by a data distribution imbalance related to data labels and data amount. In real-world applications, data are usually described by multiple modalities. However, to the best of our knowledge, only a handful of studies have been conducted to improve system performance utilizing multimodal data. In this survey paper, we identify the significance of this emerging research topic of multimodal federated learning (MFL) and present a literature review on the state-of-art MFL methods. Furthermore, we categorize multimodal federated learning into congruent and incongruent multimodal federated learning based on whether all clients possess the same modal combinations. We investigate the feasible application tasks and related benchmarks for MFL. Lastly, we summarize the promising directions and fundamental challenges in this field for future research.
Sleep is a complex and dynamic biological process characterized by different sleep patterns. Comprehensive sleep monitoring and analysis using multivariate polysomnography (PSG) records has achieved ...significant efforts to prevent sleep-related disorders. To alleviate the time consumption caused by manual visual inspection of PSG, automatic multivariate sleep stage classification has become an important research topic in medical and bioinformatics.
We present a unified hybrid self-attention deep learning framework, namely HybridAtt, to automatically classify sleep stages by capturing channel and temporal correlations from multivariate PSG records. We construct a new multi-view convolutional representation module to learn channel-specific and global view features from the heterogeneous PSG inputs. The hybrid attention mechanism is designed to further fuse the multi-view features by inferring their dependencies without any additional supervision. The learned attentional representation is subsequently fed through a softmax layer to train an end-to-end deep learning model.
We empirically evaluate our proposed HybridAtt model on a benchmark PSG dataset in two feature domains, referred to as the time and frequency domains. Experimental results show that HybridAtt consistently outperforms ten baseline methods in both feature spaces, demonstrating the effectiveness of HybridAtt in the task of sleep stage classification.
Celotno besedilo
Dostopno za:
DOBA, IZUM, KILJ, NUK, PILJ, PNG, SAZU, SIK, UILJ, UKNU, UL, UM, UPUK
The discovery of Wireless Power Transfer (WPT) technologies makes charging more convenient and reliable. Among all the existing WPT technologies, directional WPT is more efficient and has been ...successfully applied to supply energy for wireless rechargeable sensor networks (WRSNs). However, the state-of-the-art methods ignore the anisotropic energy receiving property of rechargeable sensors, resulting in energy wastage. In order to address this issue, in this paper, we point out that the received energy of a sensor is not only relative to the distance, but also relative to the angle between the sensor and the charger's orientation in directional WPT. Towards this end, we derive a pragmatic energy transfer model verified by experiments. In particular, we focus on a Minimal chArging Delay (MAD) problem to reduce chArging delays. To obtain the optimal solution, we formulate the problem as a linear programming problem. Moreover, we introduce a method of charging power discretization, which significantly reduces the search space and bounds the performance gap to the optimal one with a \displaystyle \frac{1}{1-\epsilon^{2}} approximation ratio. Besides, a merging method is introduced for a more practical application scenario. Finally, we demonstrate that our methods outperform the Set Cover baseline method by an average of 34.2% through simulations and experiments.
Diagnosis aims to predict the future health status of patients according to their historical electronic health records (EHR), which is an important yet challenging task in healthcare informatics. ...Existing diagnosis prediction approaches mainly employ recurrent neural networks (RNN) with attention mechanisms to make predictions. However, these approaches ignore the importance of code descriptions, i.e., the medical definitions of diagnosis codes. We believe that taking diagnosis code descriptions into account can help the state-of-the-art models not only to learn meaning code representations, but also to improve the predictive performance, especially when the EHR data are insufficient.
We propose a simple, but general diagnosis prediction framework, which includes two basic components: diagnosis code embedding and predictive model. To learn the interpretable code embeddings, we apply convolutional neural networks (CNN) to model medical descriptions of diagnosis codes extracted from online medical websites. The learned medical embedding matrix is used to embed the input visits into vector representations, which are fed into the predictive models. Any existing diagnosis prediction approach (referred to as the base model) can be cast into the proposed framework as the predictive model (called the enhanced model).
We conduct experiments on two real medical datasets: the MIMIC-III dataset and the Heart Failure claim dataset. Experimental results show that the enhanced diagnosis prediction approaches significantly improve the prediction performance. Moreover, we validate the effectiveness of the proposed framework with insufficient EHR data. Finally, we visualize the learned medical code embeddings to show the interpretability of the proposed framework.
Given the historical visit records of a patient, the proposed framework is able to predict the next visit information by incorporating medical code descriptions.
Celotno besedilo
Dostopno za:
DOBA, IZUM, KILJ, NUK, PILJ, PNG, SAZU, SIK, UILJ, UKNU, UL, UM, UPUK
Hospital readmission is one of the most important service quality measures. Recently, numerous risk assessment models have been proposed to address the hospital readmission problem. However, poor ...understanding of the class-imbalance hospital readmission data still challenges the development of accurate predictive models. To overcome the issue, a new risk prediction method termed joint imbalanced classification and feature selection (JICFS) is proposed for handling such a problem. To be specific, we construct the loss function within the large margin framework, in which the sample weight is involved to deal with the class imbalanced problem. Based on this, we design an optimization objective function involving ℓ1-norm regularization for improving the performance, and an iterative scheme is proposed to solve the optimization problem, thereby achieving feature selection to improve the performance. Finally, experimental results on six real-world hospital readmission datasets demonstrate that the proposed algorithm has the advantage compared with some state-of-the-art methods.
•We propose a new method, which joint imbalanced classification and feature selection, for handling class-imbalance problem.•An iterative optimization scheme is designed with convexity.•The proposed method is proved to be effective for the hospital readmission task.
Session-based recommendation is devoted to characterizing preferences of anonymous users based on short sessions. Existing methods mostly focus on mining limited item co-occurrence patterns exposed ...by item ID within sessions, while ignoring what attracts users to engage with certain items is rich multi-modal information displayed on pages. Generally, the multi-modal information can be classified into two categories: descriptive information (e.g., item images and description text) and numerical information (e.g., price). In this paper, we aim to improve session-based recommendation by modeling the above multi-modal information holistically. There are mainly three issues to reveal user intent from multi-modal information: (1) How to extract relevant semantics from heterogeneous descriptive information with different noise? (2) How to fuse these heterogeneous descriptive information to comprehensively infer user interests? (3) How to handle probabilistic influence of numerical information on user behaviors? To solve above issues, we propose a novel multi-modal session-based recommendation (MMSBR) that models both descriptive and numerical information under a unified framework. Specifically, a pseudo-modality contrastive learning is devised to enhance the representation learning of descriptive information. Afterwards, a hierarchical pivot transformer is presented to fuse heterogeneous descriptive information. Moreover, we represent numerical information with Gaussian distribution and design a Wasserstein self-attention to handle the probabilistic influence mode. Extensive experiments on three real-world datasets demonstrate the effectiveness of the proposed MMSBR. Further analysis also proves that our MMSBR can alleviate the cold-start problem in SBR effectively.
Diagnosis prediction aims to predict the patient's future diagnosis based on their Electronic Health Records (EHRs). Most existing works adopt recurrent neural networks (RNNs) to model the sequential ...EHR data. However, they mainly utilize medical codes and ignore other useful information such as patients' clinical features and demographics. We proposed a new model called MDP to augment the prediction performance by integrating the multimodal clinical data. MDP learns the clinical feature representation by adjusting the weights of clinical features based on a patient's current health condition and demographics. Also, the clinical feature representation, diagnosis codes representation and the demographic embedding are integrated to perform the prediction task. Experiments on a real-world dataset demonstrate that MDP outperforms the state-of-the-art methods.
•Provides a new optimization framework for solving the readmission prediction.•Propose a graph-based method to deal with the class-imbalanced problem.•Present an end-to-end trainable prediction model ...to improve the generalization.•Applied the proposed method on six real-world readmission datasets.•The method is proved to be effective in comparison to other methods.
Predicting hospital readmission with effective machine learning techniques has attracted a great attention in recent years. The fundamental challenge of this task stems from characteristics of the data extracted from electronic health records (EHR), which are imbalanced class distributions. This challenge further leads to the failure of most existing models that only provide a partial understanding for the learning problem and result in a biased and inaccurate prediction. To address this challenge, we propose a new graph-based class-imbalance learning method by fully making use of the data from different classes. First, we conduct graph construction for learning the pattern discrimination from between-class and within-class data samples. Then we design an optimization framework to incorporate the constructed graphs to obtain a class-imbalance aware graph embedding and further alleviate performance degeneration. Finally, we design a neural network model as the classifier to conduct imbalanced classification, i.e., hospital readmission prediction. Comprehensive experiments on six real-world readmission datasets show that the proposed method outperforms state-of-the-art approaches in readmission prediction task.
Predicting patients' risk of developing certain diseases is an important research topic in healthcare. Personalized predictive modeling, which focuses on building specific models for individual ...patients, has shown its advantages on utilizing heterogeneous health data compared to global models trained on the entire population. Personalized predictive models use information from similar patient cohorts, in order to capture the specific characteristics. Accurately identifying and ranking the similarity among patients based on their historical records is a key step in personalized modeling. The electric health records (EHRs), which are irregular sampled and have varied patient visit lengths, cannot be directly used to measure patient similarity due to lack of an appropriate vector representation. In this paper, we build a novel time fusion CNN framework to simultaneously learn patient representations and measure pairwise similarity. Compared to a traditional CNN, our time fusion CNN can learn not only the local temporal relationships but also the contributions from each time interval. Along with the similarity learning process, the output information which is the probability distribution is used to rank similar patients. Utilizing the similarity scores, we perform personalized disease predictions, and compare the effect of different vector representations and similarity learning metrics.
PDF
