Homography estimation is a basic image alignment method in many applications. It is usually done by extracting and matching sparse feature points, which are error-prone in low-light and low-texture ...images. On the other hand, previous deep homography approaches use either synthetic images for supervised learning or aerial images for unsupervised learning, both ignoring the importance of handling depth disparities and moving objects in real-world applications. To overcome these problems, in this work, we propose an unsupervised deep homography method with a new architecture design. In the spirit of the RANSAC procedure in traditional methods, we specifically learn an outlier mask to only select reliable regions for homography estimation. We calculate loss with respect to our learned deep features instead of directly comparing image content as did previously. To achieve the unsupervised training, we also formulate a novel triplet loss customized for our network. We verify our method by conducting comprehensive comparisons on a new dataset that covers a wide range of scenes with varying degrees of difficulties for the task. Experimental results reveal that our method outperforms the state-of-the-art, including deep solutions and feature-based solutions.
We discuss several techniques for mapping the gaze point to fiducial markers detected in a video stream, as commonly used in Augmented Reality eye-tracking applications. Specifically, we focus on ...using the recovered camera rotation and translation determined by the apparent distortion of the marker. We exploit this to map Areas Of Interest (AOIs) in the plane relative to the given marker. This gives the advantage of not needing any more than a single marker. We also review how the recovered homography is transformed to the graphics world coordinates so that AOI visualization can be performed in world space instead of camera space.
Pose Estimation for Augmented Reality: A Hands-On Survey Marchand, Eric; Uchiyama, Hideaki; Spindler, Fabien
IEEE transactions on visualization and computer graphics,
2016-Dec.-1, 2016-12-00, 2016-12-1, 20161201, 2016-12, Volume:
22, Issue:
12
Journal Article
Peer reviewed
Open access
Augmented reality (AR) allows to seamlessly insert virtual objects in an image sequence. In order to accomplish this goal, it is important that synthetic elements are rendered and aligned in the ...scene in an accurate and visually acceptable way. The solution of this problem can be related to a pose estimation or, equivalently, a camera localization process. This paper aims at presenting a brief but almost self-contented introduction to the most important approaches dedicated to vision-based camera localization along with a survey of several extension proposed in the recent years. For most of the presented approaches, we also provide links to code of short examples. This should allow readers to easily bridge the gap between theoretical aspects and practical implementations.
Building up on the advances in low rank matrix completion, this paper presents a novel method for propagating the inpainting of the central view of a light field to all the other views. After ...generating a set of warped versions of the inpainted central view with random homographies, both the original light field views and the warped ones are vectorized and concatenated into a matrix. Because of the redundancy between the views, the matrix satisfies a low rank assumption enabling us to fill the region to inpaint with low rank matrix completion. To this end, a new matrix completion algorithm, better suited to the inpainting application than existing methods, is also developed in this paper. In its simple form, our method does not require any depth prior, unlike most existing light field inpainting algorithms. The method has then been extended to better handle the case where the area to inpaint contains depth discontinuities. In this case, a segmentation map of the different depth layers of the inpainted central view is required. This information is used to warp the depth layers with different homographies. Our experiments with natural light fields captured with plenoptic cameras demonstrate the robustness of the low rank approach to noisy data as well as large color and illumination variations between the views of the light field.
Pose-invariant face recognition (PIFR) refers to the ability that recognizes face images with arbitrary pose variations. Among existing PIFR algorithms, pose normalization has been proved to be an ...effective approach which preserves texture fidelity, but usually depends on precise 3D face models or at high computational cost. In this paper, we propose an highly efficient PIFR algorithm that effectively handles the main challenges caused by pose variation. First, a dense grid of 3D facial landmarks are projected to each 2D face image, which enables feature extraction in an pose adaptive manner. Second, for the local patch around each landmark, an optimal warp is estimated based on homography to correct texture deformation caused by pose variations. The reconstructed frontal-view patches are then utilized for face recognition with traditional face descriptors. The homography-based normalization is highly efficient and the synthesized frontal face images are of high quality. Finally, we propose an effective approach for occlusion detection, which enables face recognition with visible patches only. Therefore, the proposed algorithm effectively handles the main challenges in PIFR. Experimental results on four popular face databases demonstrate that the propose approach performs well on both constrained and unconstrained environments.
•We propose a highly efficient and accurate pose normalization approach for pose-invariant face recognition.•This is the first time that homography is utilized for face synthesis.•The proposed approach covers the full range of pose variations within ±90° of yaw.•The proposed approach outperforms existing methods on four popular face databases.
Image stitching is a well-studied problem and has many applications in a variety of fields. Traditional feature based methods rely heavily on accurate localization or even distribution of ...hand-crafted features, and may fail for some difficult cases. Although there are robust deep learning based homography estimation or semantic alignment methods, their accuracies are not high enough for image stitching problem. In this paper, we present a deep neural network that estimates homography accurately enough for image stitching of images with small parallax. The key components of our network are feature maps with progressively increased resolution and matching cost volumes constructed in hybrid manner. Both of these designs are illustrated to be helpful for performance improvement. We also propose a new stitching oriented loss function that takes image contents into consideration. To train our network, we prepare a synthesized training dataset, the image pairs in which are more nature and similar to those of real world image stitching problem. Experimental results demonstrate that our method outperforms existing deep learning based methods and traditional feature based method in term of quantitative evaluation, visual stitching result and robustness.
Image registration is a basic task in computer vision, for its wide potential applications in image stitching, stereo vision, motion estimation, and etc. Most current methods achieve image ...registration by estimating a global homography matrix between candidate images with point-feature-based matching or direct prediction. However, as real-world 3D scenes have point-variant photograph distances (depth), a unified homography matrix is not sufficient to depict the specific pixel-wise relations between two images. Some researchers try to alleviate this problem by predicting multiple homography matrixes for different patches or segmentation areas in images; in this letter, we tackle this problem with further refinement, i.e. matching images with pixel-wise, depth-aware homography estimation. Firstly, we construct an efficient convolutional network, the DPH-Net , to predict the essential parameters causing image deviation, the rotation (<inline-formula><tex-math notation="LaTeX">R</tex-math></inline-formula>) and translation (<inline-formula><tex-math notation="LaTeX">T</tex-math></inline-formula>) of cameras. Then, we feed-in an image depth map for the calculation of initial pixel-wise homography matrixes, which are refined with an online optimization scheme. Finally, with the estimated pixel-specific homography parameters, pixel correspondences between candidate images can be easily computed for registration. Compared with state-of-the-art image registration algorithms, the proposed DPH-Net has the highest performance of 0.912 EPE and 0.977 SSIM, demonstrating the effectiveness of adding depth information and estimating pixel-wise homography into the image registration process.
A survey on image mosaicing techniques Ghosh, Debabrata; Kaabouch, Naima
Journal of visual communication and image representation,
January 2016, 2016-01-00, Volume:
34
Journal Article
Peer reviewed
•We perform a comprehensive survey of the existing image mosaicing algorithms.•We classify the state-of-the-art mosaicing techniques into major groups.•We present a comparative overview of the ...different mosaicing categories.
Image mosaicing, the process of obtaining a wider field-of-view of a scene from a sequence of partial views, has been an attractive research area because of its wide range of applications, including motion detection, resolution enhancement, monitoring global land usage, and medical imaging. A number of image mosaicing algorithms have been proposed over the last two decades. This paper provides an in-depth survey of the existing image mosaicing algorithms by classifying them into several groups. For each group, the fundamental concepts are first explained and then the modifications made to the basic concepts by different researchers are explained. Furthermore, this paper also discusses the advantages and disadvantages of all the mosaicing groups.
FRAGMENTS Abate, D.
Journal of cultural heritage,
January-February 2021, Volume:
47
Journal Article
Peer reviewed
A large number of outstanding heritage sites and artifacts around the world are fragile assets, faced with different and continuous challenges. The loss of archaeological heritage indeed continues ...today at a constant pace, both due to natural disasters and human-made actions. The magnitude of the problem requires effective measures that can support the restoration activities in case of disasters. The latter, indeed, can be difficult and time-consuming, especially when dealing with the recomposition of fragmented and scattered objects. This is often a manual procedure that does not always lead to the correct identification of the numerous possible matches. The heritage science community urges for methods able to optimize the research efforts and the fragments matching reliability and speed. The proposed workflow (FRAGMENTS) introduces a fully automatic approach based on photogrammetric techniques, given a pre-disaster image is available.
PDF
