In practical applications, it is difficult to acquire sufficient fault samples for training deep learning fault diagnosis model of rolling bearing. Aiming at the few-shot issue and multi-label ...attributes of single-point faults, a novel fault diagnosis method of rolling bearing based on time–frequency signature matrix (T–FSM) feature and multi-label convolutional neural network with meta-learning (MLCML) is proposed in this paper. At the beginning, the T–FSM features sensitive to few-shot fault diagnosis of measured vibration signal are extracted. Subsequently, a designed multi-label convolutional neural network (MLCNN) with a specific architecture is employed to identify faults. Crucially, the meta-learning strategy of learning initial network parameters susceptive to task changes is incorporated to MLCNN for addressing the few-shot problem. Ultimately, the publicly available rolling bearing dataset is utilized to demonstrate the effectiveness of the proposed method. The experimental results exhibit that the trained MLCML has the capability of learning to learn few-shot fault attributes with outstanding diagnosis accuracy and generalization. More concretely, the model can adapt to new fault categories rapidly owing to that only a few samples and update steps are required to fine-tune the network.
This paper proposes a new method of mixed gas identification based on a convolutional neural network for time series classification. In view of the superiority of convolutional neural networks in the ...field of computer vision, we applied the concept to the classification of five mixed gas time series data collected by an array of eight MOX gas sensors. Existing convolutional neural networks are mostly used for processing visual data, and are rarely used in gas data classification and have great limitations. Therefore, the idea of mapping time series data into an analogous-image matrix data is proposed. Then, five kinds of convolutional neural networks-VGG-16, VGG-19, ResNet18, ResNet34 and ResNet50-were used to classify and compare five kinds of mixed gases. By adjusting the parameters of the convolutional neural networks, the final gas recognition rate is 96.67%. The experimental results show that the method can classify the gas data quickly and effectively, and effectively combine the gas time series data with classical convolutional neural networks, which provides a new idea for the identification of mixed gases.
Dysarthric speech recognition helps speakers with dysarthria to enjoy better communication. However, collecting dysarthric speech is difficult. The machine learning models cannot be trained ...sufficiently using dysarthric speech. To further improve the accuracy of dysarthric speech recognition, we proposed a Multi-stage AV-HuBERT (MAV-HuBERT) framework by fusing the visual information and acoustic information of the dysarthric speech. During the first stage, we proposed to use convolutional neural networks model to encode the motor information by incorporating all facial speech function areas. This operation is different from the traditional approach solely based on the movement of lip in audio-visual fusion framework. During the second stage, we proposed to use the AV-HuBERT framework to pre-train the recognition architecture of fusing audio and visual information of the dysarthric speech. The knowledge gained by the pre-trained model is applied to address the overfitting problem of the model. The experiments based on UASpeech are designed to evaluate our proposed method. Compared with the results of the baseline method, the best word error rate (WER) of our proposed method was reduced by 13.5% on moderate dysarthric speech. In addition, for the mild dysarthric speech, our proposed method shows the best result that the WER of our proposed method arrives at 6.05%. Even for the extremely severe dysarthric speech, the WER of our proposed method achieves at 63.98%, which reduces by 2.72% and 4.02% compared with the WERs of wav2vec and HuBERT, respectively. The proposed method can effectively further reduce the WER of the dysarthric speech.
Endangered language generally has low-resource characteristics, as an immaterial cultural resource that cannot be renewed. Automatic speech recognition (ASR) is an effective means to protect this ...language. However, for low-resource language, native speakers are few and labeled corpora are insufficient. ASR, thus, suffers deficiencies including high speaker dependence and over fitting, which greatly harms the accuracy of recognition. To tackle the deficiencies, the paper puts forward an approach of audiovisual speech recognition (AVSR) based on LSTM-Transformer. The approach introduces visual modality information including lip movements to reduce the dependence of acoustic models on speakers and the quantity of data. Specifically, the new approach, through the fusion of audio and visual information, enhances the expression of speakers' feature space, thus achieving the speaker adaptation that is difficult in a single modality. The approach also includes experiments on speaker dependence and evaluates to what extent audiovisual fusion is dependent on speakers. Experimental results show that the CER of AVSR is 16.9% lower than those of traditional models (optimal performance scenario), and 11.8% lower than that for lip reading. The accuracy for recognizing phonemes, especially finals, improves substantially. For recognizing initials, the accuracy improves for affricates and fricatives where the lip movements are obvious and deteriorates for stops where the lip movements are not obvious. In AVSR, the generalization onto different speakers is also better than in a single modality and the CER can drop by as much as 17.2%. Therefore, AVSR is of great significance in studying the protection and preservation of endangered languages through AI.
ABSTRACTWith the technology development, industrial defect detection based on deep learning has attracted extensive attention in the academic community. Different from general visual objects, ...industrial defects have the characteristics of small sample, weak visibility and irregular shape, which hinder the application of related studies. According to these problems, a few-shot object detection (FSOD) method based on Decoupled Faster R-CNN (DeFRCN) is proposed in this paper. Firstly, it includes fine-tuning processing, because of the small sample characteristics. To adapt to the invisible characteristics of defects, we introduce the Feature Pyramid Network (FPN) and Residual Attention Module (RAM) into DeFRCN, which can enhance the capture ability of multi-scale features and feature association information. Furthermore, the feature representation ability is strengthened by parallel connecting of two channels, consisting of R-CNN head, box classifier and box regression models. Finally, it is completed that the pre-training, fine-tuning and testing of the proposed network, with DAGM 2007 and NEU-DET public industrial defect datasets as the base class and flange shaft defect data collected in the laboratory as the new class. To verify the effectiveness of the proposed one, we compare them with other classical FSOD methods. The superiority of the proposed method is obvious.
Celotno besedilo
Dostopno za:
DOBA, IZUM, KILJ, NUK, PILJ, PNG, SAZU, UILJ, UKNU, UL, UM, UPUK
Visible-infrared image fusion cannot only reveal respective features of multiband imaging but also combine complementary information. It thus highlights salient information that cannot be directly ...obtained from a single waveband and enhances scene detection and perception. However, low-light condition for special scenarios, i.e., underground coal mine, impacts the performance of visible-infrared image fusion as they lead to lower contrast for visible light images and loss of local details. In this respect, we propose an infrared and visible image fusion architecture in low-light conditions based on self- and cross-attention (SACA-Fusion). This architecture replaces traditional fusion approaches with a transformer-based fusion network. It better extracts long-range dependencies of images and improves space recovery of fused images. The architecture has an attention mechanism composed of two modules. The self-attention module achieves global interaction and fusion of features and reduces loss in local details; the cross-attention module in nest connect enhances features in low-light conditions and achieves low-contrast space recovery. In the experiment part, through ablation, we confirm that the wonderful fusion strategy is transformer module, rather than RFN or directly connecting. Then, based on comparison experiments on TNO and LLVIP datasets, it is shown that the better fusion performance of the proposed one under some evaluation indicators. Especially in the actual low-light condition, the improvement of the fusion effect is commendable.
Cross-domain few-shot classification aims to recognize images in the new categories and domains that only contain few but unacquainted images. Considering the problems of fine-grained recognition in ...cross-domain few-shot classification including marginal overall-discrepancy in feature distribution and obvious fine-grained difference in dataset, this paper proposes a simple and effective attentive fine-grained recognition (AFGR) model. Specifically, the residual attention module is stacked into the feature encoder based on the residual network, which can linearly enhance different semantic feature information to help the metric function better locate the fine-grained feature information of the image. In addition, a bilinear metric function structure is proposed to learn and fuse different fine-grained image features, respectively, since the weights of bilinear measurement functions are not shared. Eventually, the final classification result is obtained by merging the recognition of bilinear metric function through posterior probability multiplication. In this paper, ablation experiments and comparative experiments are carried out with the typical few-shot dataset mini-ImageNet as the training domain and the CUB, Cars, Places and Plantae datasets as the test domain. The experimental results demonstrate that the proposed AFGR method is effective, with the highest increase in recognition accuracy 13.82% and 7.95% compared with the latest results under the experimental settings of 5-way1-shot and 5-way5-shot, respectively, which also proves the problems of fine-grained recognition in cross-domain small sample classification.
With the development of artificial intelligence technologies, it is possible to use computer to read digital medical images. Because Alzheimer’s disease (AD) has the characteristics of high incidence ...and high disability, it has attracted the attention of many scholars, and its diagnosis and treatment have gradually become a hot topic. In this paper, a multimodal diagnosis method for AD based on three-dimensional shufflenet (3DShuffleNet) and principal component analysis network (PCANet) is proposed. First, the data on structural magnetic resonance imaging (sMRI) and functional magnetic resonance imaging (fMRI) are preprocessed to remove the influence resulting from the differences in image size and shape of different individuals, head movement, noise, and so on. Then, the original two-dimensional (2D) ShuffleNet is developed three-dimensional (3D), which is more suitable for 3D sMRI data to extract the features. In addition, the PCANet network is applied to the brain function connection analysis, and the features on fMRI data are obtained. Next, kernel canonical correlation analysis (KCCA) is used to fuse the features coming from sMRI and fMRI, respectively. Finally, a good classification effect is obtained through the support vector machines (SVM) method classifier, which proves the feasibility and effectiveness of the proposed method.
In recent years, following the development of sensor and computer techniques, it is favored by many fields, i.e. automatic drive, intelligent home, etc., which the deep learning based semantic ...segmentation method for point cloud data collected by LiDAR. This type method can automatic extract features of point cloud, helping label semantic categories. However, compared to 2D images, 3D point cloud data is more expensive to acquire. Hence, to save research and production costs, the low-thread LiDAR is a good choice. For one observation scenario, following the decrease of the line, the point cloud will become sparse, which may cause the information loss. To balance the cost and the segmentation effect, we provide a point cloud completion auxiliary semantic segmentation method. The baseline of the proposed method is Bilateral Augmentation and Adaptive Fusion (BAAF) model. The main contribution is a completion module introduction in feature extraction part of BAAF for point cloud data reconstruction. Under the premise of using low-thread LiDAR sensor to collect data, the semantic segmentation effect of 3D field point cloud is improved as much as possible. It provides theoretical basis for cost saving in practical industrial application. The feature extraction of completion module consists of Graph Aggregation Convolution (GAC) and attention mechanism. Then, we use shuffle transform to upsampling data. In addition, to analyze the effectiveness of the proposed method, we make a new dataset with sparse point cloud data, i.e. Sparse-SemanticKITTI dataset, based on public SemanticKITTI dataset. Furthermore, in experiment part, we prove the research significance. Moreover, we compare the segmentation results between classical methods to the proposed one based on point cloud data in SemanticKITTI, Sparse-SemanticKITTI and Semantic3D dataset, respectively. The effectiveness of the proposed one is obvious. Finally, the model complexity is analyzed. In sum, we provide a sparse point cloud semantic segmentation method to balance the cost and the effect.
Drift is an important issue that impairs the reliability of sensors, especially in gas sensors. The conventional method usually adopts the reference gas to compensate for the drift. However, its ...classification accuracy is not high. We propose a supervised learning algorithm that is based on multi-classifier integration for drift compensation in this paper, which incorporates drift compensation into the classification process, motivated by the fact that the goal of drift compensation is to improve the classification performance. In our method, with the obtained characteristics of sensors and the advantage of Support Vector Machine (SVM) in few-shot classification, the improved Long Shot Term Memory (LSTM) is integrated to build the multi-class classifier model. We tested the proposed approach on the publicly available time series dataset that was collected over three years by the metal-oxide gas sensors. The results clearly indicate the superiority of multiple classifier approach, which achieves higher classification accuracy as compared with different approaches during testing period with an ensemble of classifiers in the presence of sensor drift over time.
PDF
