•A novel urban terrain reconstruction method is proposed based on multisource data fusion.•The MDF method can improve the performances of the flooding simulations.•The correlation between simulated ...flooding area and elevation is enhanced.
A digital elevation model (DEM) is crucial for hydraulic/hydrodynamic simulation of urban pluvial flooding. To reduce the simulation error caused by the low resolution and poor vertical accuracy of DEM data, a novel urban terrain reconstruction method is proposed based on multisource data fusion (MDF). A 1D-2D coupled pluvial flood model under the typhoon rainfall scenario is built to verify the superiority of the proposed method. The results show that the systematic elevation error of terrain is reduced with MDF-kriging and MDF-IDW, and the performances of simulations are improved by 5% and 23% in terms of peak water-level error compared to using conventional kriging and IDW methods, and by 0.13m and 0.11m in terms of manhole water level accuracy compared to using raw DEM. Moreover, the temporospatial patterns of the flooding simulation results are more rational with the scattered flooding eliminated, and the proportion of flood area with ground elevation lower than 8 m increased from 37.9% to 92.34% and 89.33%, respectively. This method is proven to be a notably effective and low-cost DEM reconstruction method for improving the accuracy of urban pluvial flooding simulation.
Martian surface, as a typical unstructured terrain, is extremely challenging for Mars exploration missions. Commonly, Mars rovers require multiple sensors to explore such harsh environment, such as ...depth cameras, range finder and other devices. However, the onboard load, power and storage of rovers are not sufficient to achieve high-level stereoscopic perception, which can be adverse to downstream tasks such as visual navigation and scientific exploration. To this end, in this paper we propose a high-level awareness perception light-weight framework using only close-shot monocular images to implement semantic 3D reconstruction of Martian landforms. This framework consists of two parts. One is a semantic segmentation module based on the proposed real-time Mars terrain segmentation (RMTS) network to extract intra-class and inter-class contexts by local supervision. The other is a depth generation module based on a dual-encoder pix2pix network to encode the visual and semantic information of monocular images, simultaneously. To validate the proposed framework, we construct a Martian planar-stereo dataset based on AI4Mars, an open-source semantic segmentation dataset for Mars surface. It contains monocular close-up Martian images, semantic images and depth images that match each other. After training, the accuracy of proposed semantic segmentation model can reach 84.0% mIoU, with 152.2 FPS on a single RTX6000-24GB GPU. The absolute relative error of pixels in depth images between generation model and the ground truth is 0.367, while the root mean square error gets to 0.510, and the accuracy is 0.753 with 42.9 FPS. The overall environment perception scheme is with 9.5FPS.
•Focus is on the coverage path planning problem with UAV for 3D terrain reconstruction.•The aim is to obtain a path that reduces the battery consumption, minimizing the turns.•Our algorithm deals ...with both convex and non-convex regions•The algorithm can perform the coverage when complex regions are considered.•Can achieve better solutions than a previous result (using less turns).
Three-dimensional terrain reconstruction from 2D aerial images is a problem of utmost importance due its wide level of applications. It is relevant in the context of intelligent systems for disaster managements (for example to analyze a flooded area), soil analysis, earthquake crisis, civil engineering, urban planning, surveillance and defense research.
It is a two level problem, being the former the acquisition of the aerial images and the later, the 3D reconstruction. We focus here in the first problem, known as coverage path planning, and we consider the case where the camera is mounted on an unmanned aerial vehicle (UAV).
In contrast with the case when ground vehicles are used, coverage path planning for a UAV is a lesser studied problem. As the areas to cover become complex, there is a clear need for algorithms that will provide good enough solutions in affordable times, while taking into account certain specificities of the problem at hand. Our algorithm can deal with both convex and non-convex areas and their main aim is to obtain a path that reduces the battery consumption, through minimizing the number of turns.
We comment on line sweep calculation and propose improvements for the path generation and the polygon decomposition problems such as coverage alternatives and the interrupted path concept. Illustrative examples show the potential of our algorithm in two senses: ability to perform the coverage when complex regions are considered, and achievement of better solution than a published result (in terms of the number of turns used).
Martian surface, as a typical unstructured terrain, is extremely challenging for Mars exploration missions. Commonly, Mars rovers require multiple sensors to explore such harsh environments, such as ...depth cameras, range finder, and other devices. However, the onboard load, power, and storage of rovers are not sufficient to achieve high-level stereoscopic perception, which can be adverse to downstream tasks such as visual navigation and scientific exploration. To this end, in this article, we propose a high-level awareness perception lightweight framework using only close-shot monocular images to implement semantic three-dimensional (3-D) reconstruction of Martian landforms. This framework consists of two parts. One is a semantic segmentation module based on the proposed real-time Mars terrain segmentation (RMTS) network to extract intraclass and interclass contexts by local supervision. The other is a depth generation module based on a dual-encoder pix2pix network to encode the visual and semantic information of monocular images simultaneously. To validate the proposed framework, we construct a Martian planar-stereo dataset based on AI4Mars, an open-source semantic segmentation dataset for Mars surface. It contains monocular close-up Martian images, semantic images, and depth images that match each other. After training, the accuracy of the proposed semantic segmentation model can reach 84.0% mean intersection over union (mIoU), with 152.2 FPS on a single RTX6000-24GB GPU. The absolute relative error of pixels in depth images between the generation model and the ground truth is 0.367, while the root-mean-square error is 0.510, and the accuracy is 0.753 with 42.9 FPS. The overall environment perception scheme is with 9.5 FPS.
Terrain classification and feature recognition are key to the locomotion mode switching of lower limb prosthesis and also help to improve the unnatural gait of amputees. This article aims to propose ...a terrain early recognition system for lower limb prostheses. Laser range sensors and inertial measurement units (IMUs) were fixed on the lower limb, and a multi-sensor information fusion and geometric classification method for a priori identification of terrain transformations and estimation of key terrain feature parameters, including slope, height, and width. Ten healthy subjects and two hip amputees participated in the classification experiments with five terrains. The classification accuracy of the system for five terrains was 98.67% indoor and 95.33% outdoor, and the terrain classification accuracy of amputees remained consistent with that of healthy subjects. The average error of terrain feature recognition was 4.37%, including 4.52% in the amputation group, which was only 0.17% higher than that in the healthy. The system can be fixed to the lower limbs and recognize five terrains and terrain features before a gait cycle.
This research introduces a novel, highly precise, and learning-free approach to locomotion mode prediction, a technique with potential for broad applications in the field of lower-limb wearable ...robotics. This study represents the pioneering effort to amalgamate 3D reconstruction and Visual-Inertial Odometry (VIO) into a locomotion mode prediction method, which yields robust prediction performance across diverse subjects and terrains, and resilience against various factors including camera view, walking direction, step size, and disturbances from moving obstacles without the need of parameter adjustments. The proposed Depth-enhanced Visual-Inertial Odometry (D-VIO) has been meticulously designed to operate within computational constraints of wearable configurations while demonstrating resilience against unpredictable human movements and sparse features. Evidence of its effectiveness, both in terms of accuracy and operational time consumption, is substantiated through tests conducted using open-source dataset and closed-loop evaluations. Comprehensive experiments were undertaken to validate its prediction accuracy across various test conditions such as subjects, scenarios, sensor mounting positions, camera views, step sizes, walking directions, and disturbances from moving obstacles. A comprehensive prediction accuracy rate of 99.00% confirms the efficacy, generality, and robustness of the proposed method.
ABSTRACTDeep learning-based super-resolution is an essential technique for acquiring high-resolution digital elevation models (DEMs) by enhancing the spatial resolution of low-resolution DEMs. ...However, current deep learning-based approaches for DEM super-resolution lack comprehensiveness in terrain information reconstruction, resulting in the need to strengthen the rationality of terrain representation. Furthermore, the limited adaptability and extension potential of these approaches restrict their practical applicability and scope, hindering further advancement. As a solution, we introduce a broadly scalable detrending-based deep learning (DTDL) spatially explicit framework for DEM super-resolution. The framework aims to improve DEM reconstruction through data processing and augmentation. It employs detrending to distinguish between large-scale terrain trends and small-scale residuals in DEMs, thereby enhancing the neural network's capacity to learn terrain information. We integrate DTDL with classical super-resolution methods (SRCNN, EDSR, and SRGAN) and conduct experiments in the Alps, Himalayas, and Rockies. The experimental results indicate that the fusion of DTDL with deep learning-based methods enhances the accuracy of terrain reconstruction and the rationality of terrain feature representation, demonstrating strong compatibility and robustness.
The topographic skeleton is the primary expression and intuitive understanding of topographic relief. This study integrated a topographic skeleton into deep learning for terrain reconstruction. ...Firstly, a topographic skeleton, such as valley, ridge, and gully lines, was extracted from a global digital elevation model (GDEM) and Google Earth Image (GEI). Then, the Conditional Generative Adversarial Network (CGAN) was used to learn the elevation sequence information between the topographic skeleton and high-precision 5 m DEMs. Thirdly, different combinations of topographic skeletons extracted from 5 m, 12.5 m, and 30 m DEMs and a 1 m GEI were compared for reconstructing 5 m DEMs. The results show the following: (1) from the perspective of the visual effect, the 5 m DEMs generated with the three combinations (5 m DEM + 1 m GEI, 12.5 m DEM + 1 m GEI, and 30 m DEM + 1 m GEI) were all similar to the original 5 m DEM (reference data), which provides a markedly increased level of terrain detail information when compared to the traditional interpolation methods; (2) from the perspective of elevation accuracy, the 5 m DEMs reconstructed by the three combinations have a high correlation (>0.9) with the reference data, while the vertical accuracy of the 12.5 m DEM + 1 m GEI combination is obviously higher than that of the 30 m DEM + 1 m GEI combination; and (3) from the perspective of topographic factors, the distribution trends of the reconstructed 5 m DEMs are all close to the reference data in terms of the extracted slope and aspect. This study enhances the quality of open-source DEMs and introduces innovative ideas for producing high-precision DEMs. Among the three combinations, we recommend the 12.5 m DEM + 1 m GEI combination for DEM reconstruction due to its relative high accuracy and open access. In regions where a field survey of high-precision DEMs is difficult, open-source DEMs combined with GEI can be used in high-precision DEM reconstruction.
For real-world simulation, terrain models must combine various types of information on material and texture in terrain reconstruction for the three-dimensional numerical simulation of terrain. ...However, the construction of such models using the conventional method often involves high costs in both manpower and time. Therefore, this study used a convolutional neural network (CNN) architecture to classify material in multispectral remote sensing images to simplify the construction of future models. Visible light (i.e., RGB), near infrared (NIR), normalized difference vegetation index (NDVI), and digital surface model (DSM) images were examined.
This paper proposes the use of the robust U-Net (RUNet) model, which integrates multiple CNN architectures, for material classification. This model, which is based on an improved U-Net architecture combined with the shortcut connections in the ResNet model, preserves the features of shallow network extraction. The architecture is divided into an encoding layer and a decoding layer. The encoding layer comprises 10 convolutional layers and 4 pooling layers. The decoding layer contains four upsampling layers, eight convolutional layers, and one classification convolutional layer. The material classification process in this study involved the training and testing of the RUNet model. Because of the large size of remote sensing images, the training process randomly cuts subimages of the same size from the training set and then inputs them into the RUNet model for training. To consider the spatial information of the material, the test process cuts multiple test subimages from the test set through mirror padding and overlapping cropping; RUNet then classifies the subimages. Finally, it merges the subimage classification results back into the original test image.
The aerial image labeling dataset of the National Institute for Research in Digital Science and Technology (Inria, abbreviated from the French Institut national de recherche en sciences et technologies du numérique) was used as well as its configured dataset (called Inria-2) and a dataset from the International Society for Photogrammetry and Remote Sensing (ISPRS). Material classification was performed with RUNet. Moreover, the effects of the mirror padding and overlapping cropping were analyzed, as were the impacts of subimage size on classification performance. The Inria dataset achieved the optimal results; after the morphological optimization of RUNet, the overall intersection over union (IoU) and classification accuracy reached 70.82% and 95.66%, respectively. Regarding the Inria-2 dataset, the IoU and accuracy were 75.5% and 95.71%, respectively, after classification refinement. Although the overall IoU and accuracy were 0.46% and 0.04% lower than those of the improved fully convolutional network, the training time of the RUNet model was approximately 10.6 h shorter. In the ISPRS dataset experiment, the overall accuracy of the combined multispectral, NDVI, and DSM images reached 89.71%, surpassing that of the RGB images. NIR and DSM provide more information on material features, reducing the likelihood of misclassification caused by similar features (e.g., in color, shape, or texture) in RGB images. Overall, RUNet outperformed the other models in the material classification of remote sensing images. The present findings indicate that it has potential for application in land use monitoring and disaster assessment as well as in model construction for simulation systems.
•A convolutional neural network (CNN) architecture to classify material in multispectral remote sensing images to simplify future models’ construction.•The RUNet model of multiple convolutional neural network architectures for material classification.•The RUNet model is based on an improved U-Net architecture combined with the shortcut connections approach.•The encoding layer includes 10 convolution layers and 4 pooling layers.•The decoding layer has 4 upsampling layers, 8 convolution layers, and one classified convolution layer.
Craters on the lunar surface are the most direct method for the study of geological processes and are of great significance to the study of lunar evolution. In order to fill the research gap on small ...craters (diameter less than 3 m), we focus on the small craters around the moving path of the Yutu-2 lunar rover and carry out a 3D reconstruction and geometrical morphology analysis on them. First, a self-calibration model with multiple feature constraints is used to calibrate the navigation camera and obtain the internal and external parameters. Then, the sequence images with overlapping regions from neighboring stations are used to obtain the precise position of the rover through the bundle adjustment (BA) method. After that, a cross-scale cost aggregation for a stereo matching network is proposed to obtain a parallax map, which can further obtain 3D point clouds of the lunar surface. Finally, the indexes of the craters are extracted (diameter D, depth d, and depth–diameter ratio dr), and the different indicators are fitted and analyzed. The results suggest that CscaNet has an anomaly percentage value of 1.73% in the KITTI2015 dataset, and an EPE of 0.74 px in the SceneFlow dataset, both of which are superior to GC-Net, DispNet, and PSMnet, and have higher reconstruction accuracy. The correlation between D and d is high and exhibits a positive correlation, while the correlation between D and dr is low. The geometric morphology expressions of small craters fitted by using D and d are significantly different from the expressions proposed by other scholars for large craters. This study provides a priori knowledge for the subsequent Von Karmen crater survey mission in the SPA Basin.
