The fusion of high spatial resolution panchromatic (PAN) data with simultaneously acquired multispectral (MS) data with the lower spatial resolution is a hot topic, which is often called ...pansharpening. In this article, we exploit the combination of machine learning techniques and fusion schemes introduced to address the pansharpening problem. In particular, deep convolutional neural networks (DCNNs) are proposed to solve this issue. The latter is combined first with the traditional component substitution and multiresolution analysis fusion schemes in order to estimate the nonlinear injection models that rule the combination of the upsampled low-resolution MS image with the extracted details exploiting the two philosophies. Furthermore, inspired by these two approaches, we also developed another DCNN for pansharpening. This is fed by the direct difference between the PAN image and the upsampled low-resolution MS image. Extensive experiments conducted both at reduced and full resolutions demonstrate that this latter convolutional neural network outperforms both the other detail injection-based proposals and several state-of-the-art pansharpening methods.
Following the great success of deep convolutional neural networks (CNNs) in computer vision, this paper proposes a complex-valued CNN (CV-CNN) specifically for synthetic aperture radar (SAR) image ...interpretation. It utilizes both amplitude and phase information of complex SAR imagery. All elements of CNN including input-output layer, convolution layer, activation function, and pooling layer are extended to the complex domain. Moreover, a complex backpropagation algorithm based on stochastic gradient descent is derived for CV-CNN training. The proposed CV-CNN is then tested on the typical polarimetric SAR image classification task which classifies each pixel into known terrain types via supervised training. Experiments with the benchmark data sets of Flevoland and Oberpfaffenhofen show that the classification error can be further reduced if employing CV-CNN instead of conventional real-valued CNN with the same degrees of freedom. The performance of CV-CNN is comparable to that of existing state-of-the-art methods in terms of overall classification accuracy.
Alcoholism changes the structure of brain. Several somatic marker hypothesis network-related regions are known to be damaged in chronic alcoholism. Neuroimaging approach can help us better ...understanding the impairment discovered in alcohol-dependent subjects. In this research, we recruited subjects from participating hospitals. In total, 188 abstinent long-term chronic alcoholic participants (95 men and 93 women) and 191 non-alcoholic control participants (95 men and 96 women) were enrolled in our experiment via computerized diagnostic interview schedule version IV and medical history interview employed to determine whether the applicants can be enrolled or excluded. The Siemens Verio Tim 3.0 T MR scanner (Siemens Medical Solutions, Erlangen, Germany) was employed to scan the subjects. Then, we proposed a 10-layer convolutional neural network for the diagnosis based on imaging, including three advanced techniques: parametric rectified linear unit (PReLU); batch normalization; and dropout. The structure of network is fine-tuned. The results show that our method secured a sensitivity of 97.73 ± 1.04%, a specificity of 97.69 ± 0.87%, and an accuracy of 97.71 ± 0.68%. We observed the PReLU gives better performance than ordinary ReLU, clipped ReLU, and leaky ReLU. The batch normalization and dropout gained enhanced performance as batch normalization overcame the internal covariate shift and dropout got over the overfitting. The results of our proposed 10-layer CNN model show its performance better than seven state-of-the-art approaches.
Physical and/or economic constraints cause acquired seismic data to be incomplete; however, complete data are required for many subsequent seismic processing procedures. Data reconstruction is a ...crucial and long-standing topic in the exploration seismology field. We extended our previous works on deep learning (DL)-based irregularly and regularly missing 2-D data reconstruction to 3-D data. A key motivation is that the 3-D convolutional neural network (CNN) can take full advantage of the 3-D nature of the data, and the additional dimension allows more information to contribute to the data reconstruction. DL also avoids many assumptions (e.g., linearity, sparsity, and low-rank) limiting conventional nonintelligent reconstruction methods. We built an artificial neural network (ANN) based on an end-to-end U-Net encoder-decoder-style 3-D CNN. The ANN was trained on large quantities of various synthetic and field 3-D seismic data using a mean-squared-error (MSE) loss function and an Adam optimizer. We demonstrated that the developed 3-D CNN reconstruction method appears to outperform the 2-D CNN for 3-D restoration. We benchmarked the ANN's generalization capacity for recovery of irregularly and regularly sampled 3-D data on several typical seismic data sets, particularly those with high missing percentages or large gaps. An ANN trained with irregularly sampled data can be partly applied to regularly sampled cases. We investigated how a key parameter, i.e., the learning rate, can be experimentally determined. In the context of the presented examples, our methodology provided a substantial improvement over an open-source state-of-the-art rank-reduction-based approach in terms of data fidelity and efficiency.
In this paper, we investigate cooperative spectrum sensing (CSS) in a cognitive radio network (CRN) where multiple secondary users (SUs) cooperate in order to detect a primary user, which possibly ...occupies multiple bands simultaneously. Deep cooperative sensing (DCS), which constitutes the first CSS framework based on a convolutional neural network (CNN), is proposed. In DCS, instead of the explicit mathematical modeling of CSS, the strategy for combining the individual sensing results of the SUs is learned autonomously with a CNN using training sensing samples regardless of whether the individual sensing results are quantized or not. Moreover, both spectral and spatial correlation of individual sensing outcomes are taken into account such that an environment-specific CSS is enabled in DCS. Through simulations, we show that the performance of CSS can be greatly improved by the proposed DCS.
Brain-computer interface provides a new communication bridge between the human mind and devices, depending largely on the accurate classification and identification of non-invasive EEG signals. ...Recently, the deep learning approaches have been widely used in many fields to extract features and classify various types of data successfully. However, the deep learning approach requires massive data to train its neural networks, and the amount of data impacts greatly on the quality of the classifiers. This paper proposes a novel approach that combines deep learning and data augmentation for EEG classification. We applied the empirical mode decomposition on the EEG frames and mixed their intrinsic mode functions to create new artificial EEG frames, followed by transforming all EEG data into tensors as inputs of the neural network by complex Morlet wavelets. We proposed two neural networks-convolutional neural network and wavelet neural network-to train the weights and classify two classes of motor imagery signals. The wavelet neural network is a new type of neural network using wavelets to replace the convolutional layers. The experimental results show that the artificial EEG frames substantially improve the training of neural networks, and both two networks yield relatively higher classification accuracies compared to prevailing approaches. Meanwhile, we also verified the performance of our new proposed wavelet neural network model in the classification of steady-state visual evoked potentials.
Convolutional neural networks (CNNs) have been extensively applied to hyperspectral (HS) image classification tasks and achieved promising performance. However, for CNN-based HS image classification ...methods, it is hard to depict the dependencies among HS image pixels in long-range distanced positions and bands. Moreover, the limited receptive field of the convolutional layers extremely hinders the development of the CNN structure. To tackle these problems, in this article, the novel bottleneck spatial-spectral transformer (BS2T) is proposed to depict the long-range global dependencies of HS image pixels, which can be regarded as a feature extraction module for HS image classification networks. More specifically, inspired by bottleneck transformer in computer vision, for HS image feature extraction, the proposed BS2T is incorporated with a feature contraction module, a multihead spatial-spectral self-attention (MHS2A) module, and a feature expansion module. In this way, convolutional operations are replaced by the MHS2A to capture the long-range dependency of HS pixels regardless of their spatial position and distance. Meanwhile, in the MHS2A module, to highlight the spectral features of HS images, we introduce the spectral information and content spatial positional information to classical multihead self-attention to make the attention more positional aware and spectral aware. On this basis, a dual-branch HS image classification framework based on 3-D CNN and BS2T is defined for jointly extracting the local-global features of HS images. Experimental results on three public HS image classification datasets show that the proposed classification framework achieves a significant improvement when compared with the state-of-the-art methods. The source code of the proposed framework can be downloaded from https://github.com/srxlnnu/BS2T .
Pan-sharpening is a fundamental and significant task in the field of remote sensing imagery processing, in which high-resolution spatial details from panchromatic images are employed to enhance the ...spatial resolution of multispectral (MS) images. As the transformation from low spatial resolution MS image to high-resolution MS image is complex and highly nonlinear, inspired by the powerful representation for nonlinear relationships of deep neural networks, we introduce multiscale feature extraction and residual learning into the basic convolutional neural network (CNN) architecture and propose the multiscale and multidepth CNN for the pan-sharpening of remote sensing imagery. Both the quantitative assessment results and the visual assessment confirm that the proposed network yields high-resolution MS images that are superior to the images produced by the compared state-of-the-art methods.
Compute-in-Memory (CIM) implemented with Resistive-Random-Access-Memory (RRAM) crossbars is a promising approach for accelerating Convolutional Neural Network (CNN) computations. The growing size in ...the number of parameters in state-of-the-art CNN models, however, creates challenge for on-chip weight storage for CIM implementations, and CNN compression becomes a crucial topic of exploration. Tensor Train (TT) decomposition can be used to decompose a tensor into smaller ones with fewer parameters, at the cost of increased number of computations. In this work we propose a technique to minimize intermediate operations across the full convolution operation and improve hardware utilization to implement TT-CNNs in CIM systems. We first use an iterative decompose-and-fine-tune method to prepare TT-CNNs. We then propose an inter-convolutional-step reuse scheme to reduce the required operation count and post-mapping RRAM count for TT-CNN implementation in tiled-CIM architecture. We demonstrate that through proper mapping, pipelining, and reuse, effective compression ratio of 12 and 20 with 0.8% and 1.4% accuracy drop, respectively for WRN; and effective compression ratio of 6 and 11 with 0.9% and 1.2% accuracy drop for VGG8. We also show that around 30% higher hardware utilization than the original CNN format can be achieved using the proposed TT-CIM approaches.
Musculoskeletal models have been widely used for detailed biomechanical analysis to characterise various functional impairments given their ability to estimate movement variables (i.e., muscle forces ...and joint moments) which cannot be readily measured in vivo. Physics-based computational neuromusculoskeletal models can interpret the dynamic interaction between neural drive to muscles, muscle dynamics, body and joint kinematics and kinetics. Still, such set of solutions suffers from slowness, especially for the complex models, hindering the utility in real-time applications. In recent years, data-driven methods have emerged as a promising alternative due to the benefits in speedy and simple implementation, but they cannot reflect the underlying neuromechanical processes. This paper proposes a physics-informed deep learning framework for musculoskeletal modelling, where physics-based domain knowledge is brought into the data-driven model as soft constraints to penalise/regularise the data-driven model. We use the synchronous muscle forces and joint kinematics prediction from surface electromyogram (sEMG) as the exemplar to illustrate the proposed framework. Convolutional neural network (CNN) is employed as the deep neural network to implement the proposed framework. Simultaneously, the physics law between muscle forces and joint kinematics is used the soft constraint. Experimental validations on two groups of data, including one benchmark dataset and one self-collected dataset from six healthy subjects, are performed. The experimental results demonstrate the effectiveness and robustness of the proposed framework.
PDF
