Estimation of forest aboveground biomass (AGB) is crucial for various technical and scientific applications, ranging from regional carbon and bioenergy policies to sustainable forest management. ...However, passive optical remote sensing, which is the most widely used remote sensing data for retrieving vegetation parameters, is constrained by spectral saturation problems and cloud cover. On the other hand, LiDAR data, which have been extensively used to estimate forest structure attributes, cannot provide sufficient spectral information of vegetation canopies. Thus, this study aimed to develop a novel synergistic approach to estimating biomass by integrating LiDAR data with Landsat 8 imagery through a deep learning-based workflow. First the relationships between biomass and spectral vegetation indices (SVIs) and LiDAR metrics were separately investigated. Next, two groups of combined optical and LiDAR indices (i.e., COLI1 and COLI2) were designed and explored to identify their performances in biomass estimation. Finally, five prediction models, including K-nearest Neighbor, Random Forest, Support Vector Regression, the deep learning model, i.e., Stacked Sparse Autoencoder network (SSAE), and multiple stepwise linear regressions, were individually used to estimate biomass with input variables of different scenarios, i.e., (i) all the COLI1 (ACOLI1), (ii) all the COLI2 (ACOLI2), (iii) ACOLI1 and all the optical (AO) and LiDAR variables (AL), and (iv) ACOLI2, AO and AL. Results showed that univariate models with the combined optical and LiDAR indices as explanatory variables presented better modeling performance than those with either optical or LiDAR data alone, regardless of the combination mode. The SSAE model obtained the best performance compared to the other tested prediction algorithms for the forest biomass estimation. The best predictive accuracy was achieved by the SSAE model with inputs of combined optical and LiDAR variables (i.e., ACOLI1, AO and AL) that yielded an R2 of 0.935, root mean squared error (RMSE) of 15.67 Mg/ha, and relative root mean squared error (RMSEr) of 11.407%. It was concluded that the presented combined indices were simple and effective by integrating LiDAR-derived structure information with Landsat 8 spectral data for estimating forest biomass. Overall, the SSAE model with inputs of Landsat 8 and LiDAR integrated information resulted in accurate estimation of forest biomass. The presented modeling workflow will greatly facilitate future forest biomass estimation and carbon stock assessments.
The registration and integration of data from different platforms are becoming more and more important for real-scene three-dimensional (3-D) reconstruction. In urban areas, the integration of ...unmanned aerial vehicle images and terrestrial images can compensate for geometric distortions and texture blurring in models generated from single-platform images. However, it remains a question of how to maintain a high accuracy while accounting for the discrepancies between various platforms. Bundle adjustment is a crucial step in building a detailed 3-D model. However, traditional bundle adjustment is usually applied to a single platform. In the case of multiplatform data with significant differences in resolution, flight height, or viewing angles, it can lead to the issues of instability and low accuracy in solving bundle adjustment problems. This article innovatively proposes a multiplatform bundle adjustment method, which is supported by object structural information. First, the method performs patch-based matching of images from different platforms and obtains cross-platform tie points. Second, refined patches obtain object structural information by calculating the depth values and ground sampling distances of image points. Finally, multiplatform bundle adjustment is conducted using weights calculated for both object and image points based on factors obtained in the second step. The experimental results show that, in general, the proposed method can achieve the accuracy level required for practical applications. Compared with the bundle adjustment method without object structural information, the improvement of accuracy is significant, with an average improvement of 53.38% across the four datasets.
•A method for prominent expression of line features in mesh models based on images.•This method avoids the problem of robustness and incompleteness of LSM.•A new three-step clustering method for 3D ...line features is proposed.•The optimized line features of the mesh are solved using the graph cut method based on Voronoi graph.•Experiments on both simple features and complex buildings datasets validate the effectiveness of the method.
In multi-view reconstruction, mesh models are one of the most important data carriers. However, existing reconstruction methods have not adequately considered the existence of real-world edge features, resulting in insufficient utilization of geometric feature information from images. This leads to issues such as surface distortion and a lack of prominent edge features in the reconstructed mesh models. To address these challenges, this paper proposes a method of line features prominent expression during the multi-view reconstruction process. This algorithm integrates image line features and the mesh model to enhance geometric features. Specifically, in this paper, the line features on the image are first extracted, and the 2D line features are sampled and fitted to synthesize 3D line features by depth map. Then, it clusters the 3D line features and establishes minimum bounding boxes. Finally, the mesh line features are determined based on graph cut methods and are prominently expressed in the mesh model. Experimental validation demonstrates that in terms of the accuracy of line feature expression, the average reprojection pass rate for line features in five datasets reaches 93.7 %. This proves that this method significantly mitigates the issue of unclear edge features on the model surface, greatly improving the visual quality of mesh models while maintaining a high accuracy.
To produce highly detailed 3D models of architectural scenes, both aerial and terrestrial images are usually captured. However, due to the different viewpoints of each set of images, visual entities ...in cross-view images show dramatic changes. The perspective distortion makes it difficult to obtain correspondences between aerial–terrestrial image pairs. To solve this problem, a tie point matching method based on variational patch refinement is proposed. First, aero triangulation is performed on aerial images and terrestrial images, respectively; then, patches are created based on sparse point clouds. Second, the patches are optimized to be close to the surface of the object by variational patch refinement. The perspective distortion and scale difference of the terrestrial and aerial images projected onto the patches are reduced. Finally, tie points between aerial and terrestrial images can be obtained through patch-based matching. Experimental evaluations using four datasets from the ISPRS benchmark datasets and Shandong University of Science and Technology datasets reveal the satisfactory performance of the proposed method in terrestrial–aerial image matching. However, matching time is increased, because point clouds need to be generated. Occlusion in an image, such as that caused by a tree, can influence the generation of point clouds. Therefore, future research directions include the optimization of time complexity and the processing of occluded images.
With the comprehensive promotion of digital construction in China, cameras scattered throughout the country are of great significance in obtaining first-hand data. However, their potential role is ...limited due to the lack of georeference information on current surveillance cameras. Provided surveillance camera images and real scenes are combined and given georeference information, this problem can be solved, allowing cameras to generate significant social benefits. This article proposed an accurate registration method based on misalignment calibration and least squares matching between real scene and surveillance camera images to address this issue. Firstly, it is necessary to convert the navigation coordinate system from which cameras obtain data to the photogrammetric coordinate system and then solve for the misalignment and internal orientation elements of the camera. Then, accurate registration is achieved using the least squares matching on pyramid images. The experiment obtained surrounding image data of two common scenes with lens pitch angles of 45°, 55°, 65°, 75°, and 85° using the surveillance camera and obtained a 3D real scene model of each scene using a low-altitude aircraft. The experiment results show that the proposed method in this paper can achieve the expected goals of accurately matching real scene and surveillance camera images and assigning georeference information. Through extensive data analysis, the success rate and accuracy rate of registration are 98.1% and 97.06%, respectively.
In scenarios such as road detection and emergency mapping, single strip photogrammetry without ground control points (GCPs) can effectively improve data acquisition efficiency. Positioning and ...orientation system (POS) assisted single strip aerial triangulation utilizes relative orientation to construct a strip model. However, due to factors such as cumulative errors, the strip model can experience distortion. The distortion affects the accuracy of aerial triangulation solutions and can even render the results unavailable. In addition, the attitude data from POS may not be precise enough, especially the yaw angle. To address this issue, this article proposes a POS assisted aerial triangulation method for single flight strip without GCPs. This method establishes an error correction model. By utilizing the exterior orientation elements observed by the POS system, this method corrects the less accurate position and attitude parameters obtained after relative orientation. This correction process results in obtaining high precision exterior orientation elements for the images. Moreover, the error correction model maintains a constant number of parameters, regardless of the quantity of images, with only 18 parameters, thus ensuring efficient model solving. Ultimately, through experimental validation, it has been demonstrated that the method proposed in this article has improved the overall accuracy by approximately 13.22% and 17.72% compared to traditional POS assisted bundle adjustment methods under different flight strips and varying attitude angle errors, separately.
Automatic reconstruction of DSMs from satellite images is a hot issue in the field of photogrammetry. Nowadays, most state-of-the-art pipelines produce 2.5D products. In order to solve some ...shortcomings of traditional algorithms and expand the means of updating digital surface models, a DSM generation method based on variational mesh refinement of satellite stereo image pairs to recover 3D surfaces from coarse input is proposed. Specifically, the initial coarse mesh is constructed first and the geometric features of the generated 3D mesh model are then optimized by using the information of the original images, while the 3D mesh subdivision is constrained by combining the image’s texture information and projection information, with subdivision optimization of the mesh model finally achieved. The results of this method are compared qualitatively and quantitatively with those of the commercial software PCI and the SGM method. The experimental results show that the generated 3D digital surface has clearer edge contours, more refined planar textures, and sufficient model accuracy to match well with the actual conditions of the ground surface, proving the effectiveness of the method. The method is advantageous for conducting research on true 3D products in complex urban areas and can generate complete DSM products with the input of rough meshes, thus indicating it has some development prospects.
This paper proposes a novel adjustment model for mosaicking low-overlap sweeping images captured by medium-altitude unmanned aerial vehicle (UAV) with a long focal length. Commonly used methods are ...not suitable for this type of data. Three innovations are proposed to make this possible: first, building a new error model denoted by an error homograph matrix, which includes the camera parameter error, perspective center error, image attitude error, and the error of projection plane; second, the error homograph matrix is considered an unknown variable and is optimized, instead of optimizing the exterior orientation element in commonly used photogrammetry method; and third, the proposed algorithm is a global optimization, avoiding the cumulative error that appeared in traditional homographic-based methods. The results reveal that the proposed adjustment model can effectively eliminate the misalignments in the seam lines, compared to the direct homograph transformation. The numerical experiment results also demonstrate that the algorithm has perfect convergence and stability. In addition, this method is also suitable for low-altitude UAV images when the covered area can be regarded as a plane.
Epipolar rectification is of great importance for 3D modeling by using UAV (Unmanned Aerial Vehicle) images; however, the existing methods seldom consider the perspective distortion relative to ...surface planes. Therefore, an algorithm for the rectification of oblique images is proposed and implemented in detail. The basic principle is to minimize the rectified images' perspective distortion relative to the reference planes. First, this minimization problem is formulated as a cost function that is constructed by the tangent value of angle deformation; second, it provides a great deal of flexibility on using different reference planes, such as roofs and the façades of buildings, to generate rectified images. Furthermore, a reasonable scale is acquired according to the dihedral angle between the rectified image plane and the original image plane. The low-quality regions of oblique images are cropped out according to the distortion size. Experimental results revealed that the proposed rectification method can result in improved matching precision (Semi-global dense matching). The matching precision is increased by about 30% for roofs and increased by just 1% for façades, while the façades are not parallel to the baseline. In another designed experiment, the selected façades are parallel to the baseline, the matching precision has a great improvement for façades, by an average of 22%. This fully proves our proposed algorithm that elimination of perspective distortion on rectified images can significantly improve the accuracy of dense matching.
Robust and rapid image dense matching is the key to large-scale three-dimensional (3D) reconstruction for multiple Unmanned Aerial Vehicle (UAV) images. However, the following problems must be ...addressed: (1) the amount of UAV image data is very large, but ordinary computer memory is limited; (2) the patch-based multi-view stereo-matching algorithm (PMVS) does not work well for narrow-baseline cases, and its computing efficiency is relatively low, and thus, it is difficult to meet the UAV photogrammetry’s requirements of convenience and speed. This paper proposes an Image-grouping and Self-Adaptive Patch-based Multi-View Stereo-matching algorithm (IG-SAPMVS) for multiple UAV imagery. First, multiple UAV images were grouped reasonably by a certain grouping strategy. Second, image dense matching was performed in each group and included three processes. (1) Initial feature-matching consists of two steps: The first was feature point detection and matching, which made some improvements to PMVS, according to the characteristics of UAV imagery. The second was edge point detection and matching, which aimed to control matching propagation during the expansion process; (2) The second process was matching propagation based on the self-adaptive patch. Initial patches were built that were centered by the obtained 3D seed points, and these were repeatedly expanded. The patches were prevented from crossing the discontinuous terrain by using the edge constraint, and the extent size and shape of the patches could automatically adapt to the terrain relief; (3) The third process was filtering the erroneous matching points. Taken the overlap problem between each group of 3D dense point clouds into account, the matching results were merged into a whole. Experiments conducted on three sets of typical UAV images with different texture features demonstrate that the proposed algorithm can address a large amount of UAV image data almost without computer memory restrictions, and the processing efficiency is significantly better than that of the PMVS algorithm and the matching accuracy is equal to that of the state-of-the-art PMVS algorithm.
PDF
