Registration of multi-sensor data (particularly visible color sensors and infrared sensors) is a prerequisite for multimodal image analysis such as image fusion. Typically, the relationships between ...image pairs are modeled by rigid or affine transformations. However, this cannot produce accurate alignments when the scenes are not planar, for example, face images. In this paper, we propose a regularized Gaussian fields criterion for non-rigid registration of visible and infrared face images. The key idea is to represent an image by its edge map and align the edge maps by a robust criterion with a non-rigid model. We model the transformation between images in a reproducing kernel Hilbert space and a sparse approximation is applied to the transformation to avoid high computational complexity. Moreover, a coarse-to-fine strategy by applying deterministic annealing is used to overcome local convergence problems. The qualitative and quantitative comparisons on two publicly available databases demonstrate that our method significantly outperforms the state-of-the-art method with an affine model. As a result, our method will be beneficial for fusion-based face recognition.
•We analyze the robustness of Gaussian fields criterion both in theory and experiment.•The Gaussian fields criterion is generalized from rigid to the non-rigid case.•We propose a new non-rigid registration method to deal with more real-world problems.•A sparse approximation on the transformation is applied to speed up the method.•We customize and apply the proposed method to visible/thermal IR face registration.
Feature matching, which refers to establishing reliable correspondence between two sets of features (particularly point features), is a critical prerequisite in feature-based registration. In this ...paper, we propose a flexible and general algorithm, which is called locally linear transforming (LLT), for both rigid and nonrigid feature matching of remote sensing images. We start by creating a set of putative correspondences based on the feature similarity and then focus on removing outliers from the putative set and estimating the transformation as well. We formulate this as a maximum-likelihood estimation of a Bayesian model with hidden/latent variables indicating whether matches in the putative set are outliers or inliers. To ensure the well-posedness of the problem, we develop a local geometrical constraint that can preserve local structures among neighboring feature points, and it is also robust to a large number of outliers. The problem is solved by using the expectation-maximization algorithm (EM), and the closed-form solutions of both rigid and nonrigid transformations are derived in the maximization step. In the nonrigid case, we model the transformation between images in a reproducing kernel Hilbert space (RKHS), and a sparse approximation is applied to the transformation that reduces the method computation complexity to linearithmic. Extensive experiments on real remote sensing images demonstrate accurate results of LLT, which outperforms current state-of-the-art methods, particularly in the case of severe outliers (even up to 80%).
In this paper, we propose an efficient algorithm, called vector field consensus, for establishing robust point correspondences between two sets of points. Our algorithm starts by creating a set of ...putative correspondences which can contain a very large number of false correspondences, or outliers, in addition to a limited number of true correspondences (inliers). Next, we solve for correspondence by interpolating a vector field between the two point sets, which involves estimating a consensus of inlier points whose matching follows a nonparametric geometrical constraint. We formulate this a maximum a posteriori (MAP) estimation of a Bayesian model with hidden/latent variables indicating whether matches in the putative set are outliers or inliers. We impose nonparametric geometrical constraints on the correspondence, as a prior distribution, using Tikhonov regularizers in a reproducing kernel Hilbert space. MAP estimation is performed by the EM algorithm which by also estimating the variance of the prior model (initialized to a large value) is able to obtain good estimates very quickly (e.g., avoiding many of the local minima inherent in this formulation). We illustrate this method on data sets in 2D and 3D and demonstrate that it is robust to a very large number of outliers (even up to 90%). We also show that in the special case where there is an underlying parametric geometrical model (e.g., the epipolar line constraint) that we obtain better results than standard alternatives like RANSAC if a large number of outliers are present. This suggests a two-stage strategy, where we use our nonparametric model to reduce the size of the putative set and then apply a parametric variant of our approach to estimate the geometric parameters. Our algorithm is computationally efficient and we provide code for others to use it. In addition, our approach is general and can be applied to other problems, such as learning with a badly corrupted training data set.
Approaches to combine local manifold learning (LML) and the k -nearest-neighbor ( k NN) classifier are investigated for hyperspectral image classification. Based on supervised LML (SLML) and k NN, a ...new SLML-weighted k NN (SLML-W k NN) classifier is proposed. This method is appealing as it does not require dimensionality reduction and only depends on the weights provided by the kernel function of the specific ML method. Performance of the proposed classifier is compared to that of unsupervised LML (ULML) and SLML for dimensionality reduction in conjunction with the k NN (ULML- k NN and SLML- k NN). Three LML methods, locally linear embedding (LLE), local tangent space alignment (LTSA), and Laplacian eigenmaps, are investigated with these classifiers. In experiments with Hyperion and AVIRIS hyperspectral data, the proposed SLML-W k NN performed better than ULML- k NN and SLML- k NN, and the highest accuracies were obtained using weights provided by supervised LTSA and LLE.
With the development of high-resolution imagery, ship detection in optical satellite images has attracted a lot of research interest because of the broad applications in fishery management, vessel ...salvage, etc. Major challenges for this task include cloud, wave, and wake clutters, and even the variability of ship sizes. In this letter, we propose an unsupervised ship detection method toward overcoming these existing issues. Visual saliency, which focuses on highlighting salient signals from scenes, is applied to extract candidate regions followed by a homogeneous filter presented to confirm suspected ship targets with complete profiles. Then, a novel descriptor, ship histogram of oriented gradient, which characterizes the gradient symmetry of ship sides, is provided to discriminate real ships. Experimental results on numerous panchromatic satellite images demonstrate the good performance of our method compared to state-of-the-art methods.
In vector field learning, regularized kernel methods such as regularized least-squares require the number of basis functions to be equivalent to the training sample size, N. The learning process thus ...has O(N3) and O(N2) in the time and space complexity, respectively. This poses significant burden on the vector learning problem for large datasets. In this paper, we propose a sparse approximation to a robust vector field learning method, sparse vector field consensus (SparseVFC), and derive a statistical learning bound on the speed of the convergence. We apply SparseVFC to the mismatch removal problem. The quantitative results on benchmark datasets demonstrate the significant speed advantage of SparseVFC over the original VFC algorithm (two orders of magnitude faster) without much performance degradation; we also demonstrate the large improvement by SparseVFC over traditional methods like RANSAC. Moreover, the proposed method is general and it can be applied to other applications in vector field learning.
•A sparse approximation is presented for vector-valued regularized least-squares.•We derive a bound for the sparse approximation.•Based on it, we give a new robust vector field learning algorithm called SparseVFC.•We apply SparseVFC to mismatch removal and it achieves state-of-the-art performance.•The proposed method has linear time and space complexities in the scale of samples.
Remote Sensing Image Fusion Using Multiscale Mapped LS-SVM Sheng Zheng, Sheng Zheng; Wen-zhong Shi, Wen-zhong Shi; Jian Liu, Jian Liu ...
IEEE transactions on geoscience and remote sensing,
05/2008, Letnik:
46, Številka:
5
Journal Article
Recenzirano
The panchromatic (Pan) sharpening of multispectral (MS) bands is an important technique in the various applications of satellite remote sensing. This paper presents an MS Pan- sharpening method using ...the proposed multiscale mapped least-squares support vector machine (LS-SVM). Under the LS-SVM framework, the salient features underlying the image are represented by support values, and the support value transform (SVT) is developed for image information extraction. The low-resolution MS bands are resampled to the fine scale of the Pan image and sharpened by injecting the detailed features extracted from the high-resolution Pan image. The support value analysis is implemented by using a series of multiscale support value filters that are deduced from the mapped LS-SVM with multiscale Gaussian radial basis function kernels. Experiments are carried out on very high resolution QuickBird MS + Pan data. Fusion simulations on spatially degraded data, whose original MS bands are available for reference, show that the proposed MS Pan-sharpening method performs comparable to the state-of-the-art in terms of the pertained quantitative quality evaluation indexes, such as the Spectral Angle Mapper, relative dimensionless global error in synthesis (ERGAS), modulation-transfer-function-based tool and quality index (Q4), etc. The SVT is an effective tool for remote sensing image fusion.
This letter proposes a simple but effective approach to automatically learn a multilayer image feature for satellite image scene classification. Different from the hand-crafted features which are ...empirically designed but lack high generalization ability, the proposed approach can autonomously extract the data-dependent feature. The presented feature extraction algorithm is composed of two layers, and the bases of these two layers are uniformly learned by a plain K-means clustering algorithm. Coincidentally, the feature extraction performance of the aforementioned two layers is consistent with visual processing of human visual cortex. More specifically, the first layer can generate edgelike bases, which are analogous to the neuron responses of primary visual cortex (V1), and the second layer can produce cornerlike bases, which resemble the neuron responses of visual extrastriate cortical area two (V2). The proposed feature extraction approach can automatically extract not only simple structure features (e.g., edges) but also complex structure features (e.g., corners and junctions). The learned feature is further discriminated by the linear support vector machine classifier for scene classification. In order to fairly demonstrate the validity of the proposed feature extraction approach, its satellite image scene classification performance is evaluated on the public UCM-21 data set. Experimental results show that the proposed approach can outperform several recent state-of-the-art approaches.
Infrared small-target detection plays an important role in image processing for infrared remote sensing. In this letter, different from traditional algorithms, we formulate this problem as salient ...region detection, which is inspired by the fact that a small target can often attract attention of human eyes in infrared images. This visual effect arises from the discrepancy that a small target resembles isotropic Gaussian-like shape due to the optics point spread function of the thermal imaging system at a long distance, whereas background clutters are generally local orientational. Based on this observation, a new robust directional saliency-based method is proposed incorporating with visual attention theory for infrared small-target detection. Experimental results demonstrate that the proposed algorithm outperforms the state-of-the-art methods for real infrared images with various typical complex backgrounds.
Deep learning-based synthetic aperture radar (SAR) image classification is an open problem when training samples are scarce. Transfer learning-based few-shot methods are effective to deal with this ...problem by transferring knowledge from the electro-optical (EO) to the SAR domain. The performance of such methods relies on extra SAR samples, such as unlabeled novel class's samples or labeled similar classes samples. However, it is unrealistic to collect sufficient extra SAR samples in some application scenarios, namely the extreme few-shot case. In this case, the performance of such methods degrades seriously. Therefore, few-shot methods that reduce the dependence on extra SAR samples are critical. Motivated by this, a novel few-shot transfer learning method for SAR image classification in the extreme few-shot case is proposed. We propose the connection-free attention module to selectively transfer features shared between EO and SAR samples from a source network to a target network to supplement the loss of information brought by extra SAR samples. Based on the Bayesian convolutional neural network, we propose a training strategy for the extreme few-shot case, which focuses on updating important parameters, namely the accurately updating important parameters. The experimental results on the three real-SAR datasets demonstrate the superiority of our method.
PDF
