This paper presents a hypserspectral image (HSI) super-resolution method, which fuses a low-resolution HSI (LR-HSI) with a high-resolution multispectral image (HR-MSI) to get high-resolution HSI ...(HR-HSI). The proposed method first extracts the nonlocal similar patches to form a nonlocal patch tensor (NPT). A novel tensor-tensor product (t - product)-based tensor sparse representation is proposed to model the extracted NPTs. Through the tensor sparse representation, both the spectral and spatial similarities between the nonlocal similar patches are well preserved. Then, the relationship between the HR-HSI and the LR-HSI is built using t - product, which allows us to design a unified objective function to incorporate the nonlocal similarity, tensor dictionary learning, and tensor sparse coding together. Finally, alternating direction method of multipliers is used to solve the optimization problem. Experimental results on three data sets and one real data set demonstrate that the proposed method substantially outperforms the existing state-ofthe-art HSI super-resolution methods.
A novel method for anomaly detection in hyperspectral images (HSIs) is proposed based on low-rank and sparse representation. The proposed method is based on the separation of the background and the ...anomalies in the observed data. Since each pixel in the background can be approximately represented by a background dictionary and the representation coefficients of all pixels form a low-rank matrix, a low-rank representation is used to model the background part. To better characterize each pixel's local representation, a sparsity-inducing regularization term is added to the representation coefficients. Moreover, a dictionary construction strategy is adopted to make the dictionary more stable and discriminative. Then, the anomalies are determined by the response of the residual matrix. An important advantage of the proposed algorithm is that it combines the global and local structure in the HSI. Experimental results have been conducted using both simulated and real data sets. These experiments indicate that our algorithm achieves very promising anomaly detection performance.
A deep neural network (DNN)-based new pansharpening method for the remote sensing image fusion problem is proposed in this letter. Research on representation learning suggests that the DNN can ...effectively model complex relationships between variables via the composition of several levels of nonlinearity. Inspired by this observation, a modified sparse denoising autoencoder (MSDA) algorithm is proposed to train the relationship between high-resolution (HR) and low-resolution (LR) image patches, which can be represented by the DNN. The HR/LR image patches only sample from the HR/LR panchromatic (PAN) images at hand, respectively, without requiring other training images. By connecting a series of MSDAs, we obtain a stacked MSDA (S-MSDA), which can effectively pretrain the DNN. Moreover, in order to better train the DNN, the entire DNN is again trained by a back-propagation algorithm after pretraining. Finally, assuming that the relationship between HR/LR multispectral (MS) image patches is the same as that between HR/LR PAN image patches, the HR MS image will be reconstructed from the observed LR MS image using the trained DNN. Comparative experimental results with several quality assessment indexes show that the proposed method outperforms other pan-sharpening methods in terms of visual perception and numerical measures.
This paper presents a new approach for hyperspectral image classification exploiting spectral-spatial information. Under the maximum a posteriori framework, we propose a supervised classification ...model which includes a spectral data fidelity term and a spatially adaptive Markov random field (MRF) prior in the hidden field. The data fidelity term adopted in this paper is learned from the sparse multinomial logistic regression (SMLR) classifier, while the spatially adaptive MRF prior is modeled by a spatially adaptive total variation (SpATV) regularization to enforce a spatially smooth classifier. To further improve the classification accuracy, the true labels of training samples are fixed as an additional constraint in the proposed model. Thus, our model takes full advantage of exploiting the spatial and contextual information present in the hyperspectral image. An efficient hyperspectral image classification algorithm, named SMLR-SpATV, is then developed to solve the final proposed model using the alternating direction method of multipliers. Experimental results on real hyperspectral data sets demonstrate that the proposed approach outperforms many state-of-the-art methods in terms of the overall accuracy, average accuracy, and kappa (k) statistic.
In this paper, we propose a variational Bayesian method for Retinex to simulate and interpret how the human visual system perceives color. To construct a hierarchical Bayesian model, we use the Gibbs ...distributions as prior distributions for the reflectance and the illumination, and the gamma distributions for the model parameters. By assuming that the reflection function is piecewise continuous and illumination function is spatially smooth, we define the energy functions in the Gibbs distributions as a total variation function and a smooth function for the reflectance and the illumination, respectively. We then apply the variational Bayes approximation to obtain the approximation of the posterior distribution of unknowns so that the unknown images and hyperparameters are estimated simultaneously. Experimental results demonstrate the efficiency of the proposed method for providing competitive performance without additional information about the unknown parameters, and when prior information is added the proposed method outperforms the non-Bayesian-based Retinex methods we compared.
The total variation model proposed by Rudin, Osher and Fatemi performs very well for removing noise while preserving edges. However, it favors a piecewise constant solution in BV space which often ...leads to the staircase effect, and small details such as textures are often filtered out with noise in the process of denoising. To preserve the textures and eliminate the staircase effect, we improve the total variation model in this paper. This is accomplished by the following steps: (1) we define a new space of functions of fractional-order bounded variation called the
BV
α
space by using the Grünwald–Letnikov definition of fractional-order derivative; (2) we model the structure of the image as a function belonging to the
BV
α
space, and the textures in different scales as functions belonging to different negative Sobolev spaces. Thus, we propose a class of fractional-order multi-scale variational models for image denoising. (3) We analyze some properties of the fraction-order total variation operator and its conjugate operator. By using these properties, we develop an alternation projection algorithm for the new model and propose an efficient condition of the convergence of the algorithm. The numerical results show that the fractional-order multi-scale variational model can improve the peak signal to noise ratio of image, preserve textures and eliminate the staircase effect efficiently in the process of denoising.
Weakly supervised object detection (WSOD) aims to predict a set of bounding boxes and corresponding category labels for instances with only image-level supervisions. Compared with fully supervised ...object detection, WSOD in remote sensing images (RSIs) is much more challenging due to the vast foreground-related context regions. In this paper, we propose a progressive image-level and instance-level feature refinement network to address the problems of missing detection and part domination for WSOD in RSIs. Firstly, we propose a multi-label attention mining loss (MAML)-guided image-level feature refinement branch to effectively allocate the computational resources towards the most informative part of images. With the supervision of MAML, all latent instances in images are emphasized. However, image-level feature refinement further expands responsive gaps between the informative part and other sub-optimal informative ones, which results in exacerbating the problem of part domination. In order to alleviate the above-mentioned limitation, we further construct an instance-level feature refinement branch to re-balance the contributions of different adjacent candidate bounding boxes according to the detection task. An instance selection loss (ISL) is proposed to progressively boost the representation of salient regions by exploring supervision from the network itself. Finally, we integrate the image-level and instance-level feature refinement branches into a complete network and the proposed MAML and ISL functions are merged with class classification and box regression to optimize the whole WSOD network in an end-to-end training fashion. We conduct experiments on two popular WSOD datasets, NWPU VHR-10.v2 and DIOR. All the experimental results demonstrate that our method achieves a competitive performance compared with other state-of-the-art approaches.
Hyperspectral image (HSI) classification accuracy has been greatly improved by employing deep learning. The current research mainly focuses on how to build a deep network to improve the accuracy. ...However, these networks tend to be more complex and have more parameters, which makes the model difficult to train and easy to overfit. Therefore, we present a lightweight deep convolutional neural network (CNN) model called S2FEF-CNN. In this model, three S2FEF blocks are used for the joint spectral–spatial features extraction. Each S2FEF block uses 1D spectral convolution to extract spectral features and 2D spatial convolution to extract spatial features, respectively, and then fuses spectral and spatial features by multiplication. Instead of using the full connected layer, two pooling layers follow three blocks for dimension reduction, which further reduces the training parameters. We compared our method with some state-of-the-art HSI classification methods based on deep network on three commonly used hyperspectral datasets. The results show that our network can achieve a comparable classification accuracy with significantly reduced parameters compared to the above deep networks, which reflects its potential advantages in HSI classification.
Hyperspectral image (HSI) super-resolution aims at improving the spatial resolution of HSI by fusing a high spatial resolution multispectral image (MSI). To preserve local submanifold structures in ...HSI super-resolution, a novel superpixel graph-based super-resolution method is proposed. Firstly, the MSI is segmented into superpixel blocks to form two-directional feature tensors, then two graphs are created using spectral–spatial distance between the unfolded feature tensors. Secondly, two graph Laplacian terms involving underlying BTD factors of high-resolution HSI are developed, which ensures the inheritance of the spatial geometric structures. Finally, by incorporating graph Laplacian priors with the coupled BTD degradation model, a HSI super-resolution model is established. Experimental results demonstrate that the proposed method achieves better fused results compared with other advanced super-resolution methods, especially on the improvement of the spatial structure.
Very high resolution (VHR) images change detection plays an important role in many remote sensing applications, such as military reconnaissance, urban planning and natural resource monitoring. ...Recently, fully connected conditional random field (FCCRF)-facilitated deep convolutional neural networks have shown promising results in change detection. However, the FCCRF in change detection currently is still postprocessing based on the output of the front-end network, which is not a convenient end-to-end network model and cannot combine front-end network knowledge with the knowledge of pairwise potential. Therefore, we propose a new end-to-end deep Siamese pairwise potential CRFs network (PPNet) for VHR images change detection. Specifically, this method adds a conditional random field recurrent neural network (CRF-RNN) unit into the convolutional neural network and integrates the knowledge of unary potential and pairwise potential in the end-to-end training process, aiming to refine the edges of changed areas and to remove the distant noise. In order to correct the front-end network identification errors, the method uses effective channel attention (ECA) to further effectively distinguish the change areas. Our experimental results on two data sets verify that the proposed method has more advanced capability with almost no increase in the number of parameters and effectively avoids the overfitting phenomenon in the training process.
PDF
