Robot manipulators are playing increasingly significant roles in scientific researches and engineering applications in recent years. Using manipulators to save labors and increase accuracies are ...becoming common practices in industry. Neural networks, which feature high-speed parallel distributed processing, and can be readily implemented by hardware, have been recognized as a powerful tool for real-time processing and successfully applied widely in various control systems. Particularly, using neural networks for the control of robot manipulators have attracted much attention and various related schemes and methods have been proposed and investigated. In this paper, we make a review of research progress about controlling manipulators by means of neural networks. The problem foundation of manipulator control and the theoretical ideas on using neural network to solve this problem are first analyzed and then the latest progresses on this topic in recent years are described and reviewed in detail. Finally, toward practical applications, some potential directions possibly deserving investigation in controlling manipulators by neural networks are pointed out and discussed.
This article proposes an end-to-end method based on an improved convolutional neural network model for inverter fault diagnosis. First, transient time-domain sequence data under different faults are ...analyzed, and raw signals are taken as fault representations without manually selecting feature extraction methods. Second, the model can automatically learn and extract features in the input domain using stacked convolution layers with the wide first-layer convolution kernel and a global max pooling layer; thus, it eliminated the influence of expert experience. Finally, the fault diagnosis results of the three-phase voltage-source inverter are automatically obtained in the softmax layer. The proposed fault diagnosis method has superior recognition performance with mixed noise data and variable load data. Contrastive experiments show that the improved fault diagnosis model is effective than traditional machine learning and other deep learning methods.
Following the great success of deep convolutional neural networks (CNNs) in computer vision, this paper proposes a complex-valued CNN (CV-CNN) specifically for synthetic aperture radar (SAR) image ...interpretation. It utilizes both amplitude and phase information of complex SAR imagery. All elements of CNN including input-output layer, convolution layer, activation function, and pooling layer are extended to the complex domain. Moreover, a complex backpropagation algorithm based on stochastic gradient descent is derived for CV-CNN training. The proposed CV-CNN is then tested on the typical polarimetric SAR image classification task which classifies each pixel into known terrain types via supervised training. Experiments with the benchmark data sets of Flevoland and Oberpfaffenhofen show that the classification error can be further reduced if employing CV-CNN instead of conventional real-valued CNN with the same degrees of freedom. The performance of CV-CNN is comparable to that of existing state-of-the-art methods in terms of overall classification accuracy.
As an important task in the field of remote sensing image processing, remote sensing image change detection (CD) has made significant advances through the use of convolutional neural networks (CNN). ...The Transformer has recently been introduced into the field of CD due to its excellent global perception capabilities. Some works have attempted to combine CNN and Transformer to jointly harvest local-global features. However, these works have not paid much attention to the interaction between the features extracted by both. Also, the use of the Transformer has resulted in significant resource consumption. In this paper, we propose the Asymmetric Cross-attention Hierarchical Network (ACAHNet) by combining CNN and Transformer in a series-parallel manners. The proposed Asymmetric Multi-headed Cross Attention (AMCA) module reduces the quadratic computational complexity of the Transformer to linear, and the module enhances the interaction between features extracted from the CNN and the Transformer. Different from the early and late fusion strategies employed in previous work, the effectiveness of the mid-term fusion strategy employed by ACAHNet shows a new choice of timing for feature fusion in the CD task. Our experiments on the proposed method on three public datasets show that our network has better performance in terms of effectiveness and computational resource consumption compared to other comparative methods.
Motor fault diagnosis is imperative to enhance the reliability and security of industrial systems. However, since motors are often operated under nonstationary conditions, the high complexity of ...vibration signals raises notable difficulties for fault diagnosis. Therefore, considering the special physical characteristics of motor signals under nonstationary conditions, in this article, we propose a multiscale kernel based residual convolutional neural network (CNN) for motor fault diagnosis. Our contributions mainly fall into two aspects. First, we notice that each motor fault category has various patterns in vibration signals due to the changing operational conditions of the motor. To capture these patterns, a multiscale kernel algorithm is applied in the CNN architecture. Second, since the motor vibration signals are made up of many different components from different transfer paths, they are very complex and variable. To enable the architecture to extract fault features from deep and hierarchical representation spaces, sufficient depth of the network is needed, which will lead to the degradation problem. In the proposed method, residual learning is embedded into the multiscale kernel CNN to avoid performance degradation and build a deeper network. To validate the effectiveness of the proposed networks, a normal motor and five motors with different failures are tested. The results and comparisons with state-of-the-art methods highlight the superiority of the proposed method.
The artificial neural network (ANN)-a machine learning technique inspired by the human neuronal synapse system-was introduced in the 1950s. However, the ANN was previously limited in its ability to ...solve actual problems, due to the vanishing gradient and overfitting problems with training of deep architecture, lack of computing power, and primarily the absence of sufficient data to train the computer system. Interest in this concept has lately resurfaced, due to the availability of big data, enhanced computing power with the current graphics processing units, and novel algorithms to train the deep neural network. Recent studies on this technology suggest its potentially to perform better than humans in some visual and auditory recognition tasks, which may portend its applications in medicine and healthcare, especially in medical imaging, in the foreseeable future. This review article offers perspectives on the history, development, and applications of deep learning technology, particularly regarding its applications in medical imaging.
•We reviewed the state-of-the-art on classification of AD based on CNN and T1 MRI.•We unveiled data leakage, leading to biased results, in some reviewed studies.•We proposed a framework for ...reproducible evaluation of AD classification methods.•We demonstrated the use of the proposed framework on three public datasets.•We assessed generalizability both within a dataset and between datasets.
Numerous machine learning (ML) approaches have been proposed for automatic classification of Alzheimer's disease (AD) from brain imaging data. In particular, over 30 papers have proposed to use convolutional neural networks (CNN) for AD classification from anatomical MRI. However, the classification performance is difficult to compare across studies due to variations in components such as participant selection, image preprocessing or validation procedure. Moreover, these studies are hardly reproducible because their frameworks are not publicly accessible and because implementation details are lacking. Lastly, some of these papers may report a biased performance due to inadequate or unclear validation or model selection procedures. In the present work, we aim to address these limitations through three main contributions. First, we performed a systematic literature review. We identified four main types of approaches: i) 2D slice-level, ii) 3D patch-level, iii) ROI-based and iv) 3D subject-level CNN. Moreover, we found that more than half of the surveyed papers may have suffered from data leakage and thus reported biased performance. Our second contribution is the extension of our open-source framework for classification of AD using CNN and T1-weighted MRI. The framework comprises previously developed tools to automatically convert ADNI, AIBL and OASIS data into the BIDS standard, and a modular set of image preprocessing procedures, classification architectures and evaluation procedures dedicated to deep learning. Finally, we used this framework to rigorously compare different CNN architectures. The data was split into training/validation/test sets at the very beginning and only the training/validation sets were used for model selection. To avoid any overfitting, the test sets were left untouched until the end of the peer-review process. Overall, the different 3D approaches (3D-subject, 3D-ROI, 3D-patch) achieved similar performances while that of the 2D slice approach was lower. Of note, the different CNN approaches did not perform better than a SVM with voxel-based features. The different approaches generalized well to similar populations but not to datasets with different inclusion criteria or demographical characteristics. All the code of the framework and the experiments is publicly available: general-purpose tools have been integrated into the Clinica software (www.clinica.run) and the paper-specific code is available at: https://github.com/aramis-lab/AD-DL.
Display omitted
Artificial neural networks (ANNs) have recently made a significant impact on the field of industrial electronics. Increasing efforts have been focused on implementing ANN using more efficient and ...compact architectures to achieve improved computational performance and power efficiency. The activation function of the nonlinear layers is crucial for the successful implementation of ANN, as it enables the network to possess generalization ability. In this study, we implemented the hyperbolic tangent function (tanh) using an analog circuit based on phase-change memory (PCM), which is a mature nonvolatile technology. This fully analog PCM-based tanh function is naturally compatible with analog-memory-based ANN architectures, such as the memristive neural networks (MNNs). An analog circuit has been proposed to implement the tanh by transforming the PCM I-V characteristics. The results indicate that the implemented tanh based on PCM presents a low root-mean-square error to the ideal tanh. Furthermore, the performance of the implemented tanh was verified in a classical deep neural network (DNN) with the handwritten digit recognition task. The implemented tanh with realistic characteristics presents a training accuracy (>90%) and good precision for repeatability inference in DNN. This work provides effective means for implementing activation functions in ANN based on analog memory.
In recent years, vector-based machine learning algorithms, such as random forests, support vector machines, and 1-D convolutional neural networks, have shown promising results in hyperspectral image ...classification. Such methodologies, nevertheless, can lead to information loss in representing hyperspectral pixels, which intrinsically have a sequence-based data structure. A recurrent neural network (RNN), an important branch of the deep learning family, is mainly designed to handle sequential data. Can sequence-based RNN be an effective method of hyperspectral image classification? In this paper, we propose a novel RNN model that can effectively analyze hyperspectral pixels as sequential data and then determine information categories via network reasoning. As far as we know, this is the first time that an RNN framework has been proposed for hyperspectral image classification. Specifically, our RNN makes use of a newly proposed activation function, parametric rectified tanh (PRetanh), for hyperspectral sequential data analysis instead of the popular tanh or rectified linear unit. The proposed activation function makes it possible to use fairly high learning rates without the risk of divergence during the training procedure. Moreover, a modified gated recurrent unit, which uses PRetanh for hidden representation, is adopted to construct the recurrent layer in our network to efficiently process hyperspectral data and reduce the total number of parameters. Experimental results on three airborne hyperspectral images suggest competitive performance in the proposed mode. In addition, the proposed network architecture opens a new window for future research, showcasing the huge potential of deep recurrent networks for hyperspectral data analysis.
Intelligent fault diagnosis methods based on signal analysis have been widely used for bearing fault diagnosis. These methods use a pre-determined transformation (such as empirical mode ...decomposition, fast Fourier transform, discrete wavelet transform) to convert time-series signals into frequency domain signals, the performance of dignostic system is significantly rely on the extracted features. However, extracting signal characteristic is fairly time consuming and depends on specialized signal processing knowledge. Although some studies have developed highly accurate algorithms, the diagnostic results rely heavily on large data sets and unreliable human analysis. This study proposes an automatic feature learning neural network that utilizes raw vibration signals as inputs, and uses two convolutional neural networks with different kernel sizes to automatically extract different frequency signal characteristics from raw data. Then long short-term memory was used to identify the fault type according to learned features. The data is down-sampled before inputting into the network, greatly reducing the number of parameters. The experiment shows that the proposed method can not only achieve 98.46% average accuracy, exceeding some state-of-the-art intelligent algorithms based on prior knowledge and having better performance in noisy environments.
PDF
