We propose a geometry-guided neural network architecture for robust and detail-preserving surface normal estimation for unstructured point clouds. Previous deep normal estimators usually estimate the ...normal directly from the neighbors of a query point, which lead to poor performance. The proposed network is composed of a weight learning sub-network (WL-Net) and a lightweight normal learning sub-network (NL-Net). WL-Net first predicates point-wise weights for generating an optimized point set (OPS) from the input. Then, NL-Net estimates a more accurate normal from the OPS especially when the local geometry is complex. To boost the weight learning ability of the WL-Net, we introduce two geometric guidance in the network. First, we design a weight guidance using the deviations between the neighbor points and the ground truth tangent plane of the query point. This deviation guidance offers a “ground truth” for weights corresponding to some reliable inliers and outliers determined by the tangent plane. Second, we integrate the normals of multiple scales into the input. Its performance and robustness are further improved without relying on multi-branch networks, which are employed in previous multi-scale normal estimators. Thus our method is more efficient. Qualitative and quantitative evaluations demonstrate the advantages of our approach over the state-of-the-art methods, in terms of estimation accuracy, model size and inference time. Code is available at https://github.com/2429581027/local-geometric-guided.
•A new two-step normal estimation method.•Integrate geometric priors into deep learning framework.•Replace multi-scale architecture by multi-scale geometric input.•Achieve 10.79 angle error in comparison with the previous state of the art of 11.78.
•A new classification method is proposed for terrestrial point clouds of soil and rock.•The method improves accuracy for snow and talus and adapts to seasonal variability.•Machine learning based and ...masking classification methods are compared.•Choosing a classification method depends on several factors other than accuracy.
High-resolution remote monitoring of slopes using terrestrial LiDAR and photogrammetry is a valuable tool for the management of civil and mining geotechnical asset hazards, but accurately classifying regions of interest in the data is sometimes a difficult and time-consuming task. Filtering unwanted areas of a point cloud, such as vegetation and talus, is often a necessary step before rockfall change detection results can be further processed into actionable information. In addition, long-term monitoring through seasonal vegetation changes and snow presents unique challenges to the goal of accurate classification in an automated workflow. This study presents a Random Forest machine learning approach to improve the classification accuracy and efficiency of terrestrial LiDAR monitoring of complex natural slopes. The algorithm classifies points as vegetation, talus, snow, and bedrock using multi-scale neighborhood geometry, slope, change, and intensity features. The classifier was trained on two manually labeled scans from summer and winter, then tested on three other unseen times. We find that F Score generally remains above 0.9 for talus and vegetation, and above 0.95 for bedrock and snow, indicating very high accuracy and an ability to adapt to changing seasonal conditions. In comparing this approach to CANUPO, an existing classification tool, we find our approach to be generally more accurate and flexible, at the expense of increased complexity and computation time. Comparisons with manual masking and a hybrid approach indicate that a machine learning solution is useful primarily in cases of rapidly changing rock slopes or in climates with significant seasonal variability and snow.
Dam deformation monitoring can directly identify the safe operation state of a dam in advance, which plays an important role in dam safety management. Three-dimensional (3D) terrestrial laser ...scanning technology is widely used in the field of deformation monitoring due to its fast, complete, and high-density 3D data acquisition capabilities. However, 3D point clouds are characterized by rough surfaces, discrete distributions, which affect the accuracy of deformation analysis of two states data. In addition, it is impossible to directly extract the correspondence points from an irregularly distributed point cloud to unify the coordinates of the two states’ data, and the correspondence lines and planes are often difficult to obtain in the natural environment. To solve the above problems, this paper studies a displacement change detection method for arch dams based on two-step point cloud registration and contour model comparison method. In the environment around a dam, the stable rock is used as the correspondence element to improve the registration accuracy, and a two-step registration method from rough to fine using the iterative closest point algorithm is present to describe the coordinate unification of the two states’ data without control network and target. Then, to analyze the displacement variation of an arch dam surface in two states and improve the accuracy of comparing the two surfaces without being affected by the roughness of the point cloud, the contour model fitting the point clouds is used to compare the change in distance between models. Finally, the method of this paper is applied to the Xiahuikeng Arch Dam, and the displacement changes of the entire dam in different periods are visualized by comparing with the existing methods. The results show that the displacement change in the middle area of the dam is generally greater than that of the two banks, increasing with the increase in elevation, which is consistent with the displacement change behavior of the arch dam during operation and can reach millimeter-level accuracy.
Recently, the advancement of deep learning (DL) in discriminative feature learning from 3-D LiDAR data has led to rapid development in the field of autonomous driving. However, automated processing ...uneven, unstructured, noisy, and massive 3-D point clouds are a challenging and tedious task. In this article, we provide a systematic review of existing compelling DL architectures applied in LiDAR point clouds, detailing for specific tasks in autonomous driving, such as segmentation, detection, and classification. Although several published research articles focus on specific topics in computer vision for autonomous vehicles, to date, no general survey on DL applied in LiDAR point clouds for autonomous vehicles exists. Thus, the goal of this article is to narrow the gap in this topic. More than 140 key contributions in the recent five years are summarized in this survey, including the milestone 3-D deep architectures, the remarkable DL applications in 3-D semantic segmentation, object detection, and classification; specific data sets, evaluation metrics, and the state-of-the-art performance. Finally, we conclude the remaining challenges and future researches.
We study the problem of efficient semantic segmentation of large-scale 3D point clouds. By relying on expensive sampling techniques or computationally heavy pre/post-processing steps, most existing ...approaches are only able to be trained and operate over small-scale point clouds. In this paper, we introduce RandLA-Net , an efficient and lightweight neural architecture to directly infer per-point semantics for large-scale point clouds. The key to our approach is to use random point sampling instead of more complex point selection approaches. Although remarkably computation and memory efficient, random sampling can discard key features by chance. To overcome this, we introduce a novel local feature aggregation module to progressively increase the receptive field for each 3D point, thereby effectively preserving geometric details. Comparative experiments show that our RandLA-Net can process 1 million points in a single pass up to 200× faster than existing approaches. Moreover, extensive experiments on five large-scale point cloud datasets, including Semantic3D, SemanticKITTI, Toronto3D, NPM3D and S3DIS, demonstrate the state-of-the-art semantic segmentation performance of our RandLA-Net.
•A method was proposed for 3D shape reconstruction from impaired point clouds.•Misaligned point clouds were unusable data but we fixed it.•A mini survey on impairments in 3D human body scans was ...proposed.
Accurate 3-D models of human subjects are widely used in domains such as fashion design, non-contact body biometrics, computer animation, gaming, AR/VR, to cite a few. For these kinds of applications, a high-fidelity human body mesh in a canonical posture (e.g. pose or pose) is necessary. This paper proposes a deep learning approach to jointly reconstruct a clean, watertight body mesh and to normalize the posture of the human body model starting from an input set of impaired body point clouds. The proposed method, dubbed Impaired-to-High-fidelity human body network (I2H) is, to the best of our knowledge, the first deep learning approach in the literature that addresses these problems. The proposed method follows an Encoder-Decoder design. The Encoder directly takes the impaired point clouds (e.g. containing noise, occlusions and misalignments) as input without making any structural assumptions about the input. The Decoder interprets the latent feature and produces a high-fidelity T-pose body mesh. We compare the proposed approach against existing state-of-the-art methods through various experiments and show that our method achieves the best performance on both synthetic and scanned datasets for 3D human mesh reconstruction.
In this paper we present an automatic algorithm to detect basic shapes in unorganized point clouds. The algorithm decomposes the point cloud into a concise, hybrid structure of inherent shapes and a ...set of remaining points. Each detected shape serves as a proxy for a set of corresponding points. Our method is based on random sampling and detects planes, spheres, cylinders, cones and tori. For models with surfaces composed of these basic shapes only, for example, CAD models, we automatically obtain a representation solely consisting of shape proxies. We demonstrate that the algorithm is robust even in the presence of many outliers and a high degree of noise. The proposed method scales well with respect to the size of the input point cloud and the number and size of the shapes within the data. Even point sets with several millions of samples are robustly decomposed within less than a minute. Moreover, the algorithm is conceptually simple and easy to implement. Application areas include measurement of physical parameters, scan registration, surface compression, hybrid rendering, shape classification, meshing, simplification, approximation and reverse engineering.
Precisely estimating a robot’s pose in a prior, global map is a fundamental capability for mobile robotics, e.g., autonomous driving or exploration in disaster zones. This task, however, remains ...challenging in unstructured, dynamic environments, where local features are not discriminative enough and global scene descriptors only provide coarse information. We therefore present SegMap: a map representation solution for localization and mapping based on the extraction of segments in 3D point clouds. Working at the level of segments offers increased invariance to view-point and local structural changes, and facilitates real-time processing of large-scale 3D data. SegMap exploits a single compact data-driven descriptor for performing multiple tasks: global localization, 3D dense map reconstruction, and semantic information extraction. The performance of SegMap is evaluated in multiple urban driving and search and rescue experiments. We show that the learned SegMap descriptor has superior segment retrieval capabilities, compared with state-of-the-art handcrafted descriptors. As a consequence, we achieve a higher localization accuracy and a 6% increase in recall over state-of-the-art handcrafted descriptors. These segment-based localizations allow us to reduce the open-loop odometry drift by up to 50%. SegMap is open-source available along with easy to run demonstrations.
Digital twins (DTs) have been found useful in manufacturing, construction, and maintenance. Adapting DTs to serve cities, the question arises of what an urban digital twin should contain and how it ...should be orchestrated to serve a city’s dynamical ecosystem, along with how to enhance the efficiency of the city. We are aligning with the commonplace idea that the main advantage of using DTs is economical as, for example, DTs can improve the planning of activities thus saving money and time. But how can they be useful for a city? Instead of looking at the DTs as solutions in search of problems to be solved, we start from city needs. Our approach is two-fold. We start by briefly reviewing existing possibilities for meeting some specific needs, but keep the focus on identifying and attempting to close the gap between the needs arising from everyday city functions and the latest DT techniques useful for meeting those needs. DTs are technically different and serve different applications, yet they share a common identity and name, as well as several technical similarities. Adopting computer science terminology, we see a back-end city DT as the container of all information, while any single front-end, visualized or used either by humans or robots, offers a limited but meaningful representation of the DT for a specific application. Alas, there are multiple open questions regarding the realization and benefits of such back-end DT. Nevertheless, we discuss how the back-end DT (or any specific DT) could be updated autonomously from sensor data using artificial intelligence techniques, and how the front-ends could be used for large benefits to the entire city ecosystem.
•There is a call to better match DT technology to meet overall city needs.•City DTs differ from DTs used in manufacturing, construction, and maintenance.•Differences include both technical (BIM-GIS) and human factor-induced complexities.•Novel AI methods could serve in automated updating of city DTs from sensor data.•Human factors and inclusion of third parties need to be considered for city DTs.
Reconstruction and expansion, as well as asset management, of highways necessitate the development of a current and highly precise 3D pavement model. Current inverse modeling methods with point ...clouds are laborious, time-consuming, and limited in precision. This article introduces an alternative framework for parametric inverse procedural modeling of highway pavement with standardized alignments seamlessly integrated with off-the-shelf modeling software. It comprises three key steps. (1) Extraction of highway pavement boundaries and lane markings: Initially, we combine grid-based and model-driven methods, followed by line structure-based clustering, to accurately generate road centerlines and layouts. (2) Road centerline generation: The centerline, derived from lane markings, informs highway alignments and parameters based on geometric characteristics such as curvature and slope. We utilize cost functions to facilitate this process. (3) Novel inverse procedural assembly: This innovative step integrates off-the-shelf modeling software. This approach involves extracting vector lines from point clouds and applying constraints at pivotal points on highway pavement cross-sections. Our focus is on refined component-level modeling, allowing for the assembly of diverse highway elements. This method significantly reduces human intervention and achieves high precision. In tests on two highway datasets from Sichuan Province, China, our method achieved excellent results. It attained an average correctness of 98.63% and completeness of 99.66% within a 10 cm error margin. A comparison with the intersection point method indicated minimal errors, with maximum values below 1.2%. The resultant 3D highway pavement model is modular and highly accurate at the centimeter level.
PDF
