Recently, the advancement of deep learning (DL) in discriminative feature learning from 3-D LiDAR data has led to rapid development in the field of autonomous driving. However, automated processing ...uneven, unstructured, noisy, and massive 3-D point clouds are a challenging and tedious task. In this article, we provide a systematic review of existing compelling DL architectures applied in LiDAR point clouds, detailing for specific tasks in autonomous driving, such as segmentation, detection, and classification. Although several published research articles focus on specific topics in computer vision for autonomous vehicles, to date, no general survey on DL applied in LiDAR point clouds for autonomous vehicles exists. Thus, the goal of this article is to narrow the gap in this topic. More than 140 key contributions in the recent five years are summarized in this survey, including the milestone 3-D deep architectures, the remarkable DL applications in 3-D semantic segmentation, object detection, and classification; specific data sets, evaluation metrics, and the state-of-the-art performance. Finally, we conclude the remaining challenges and future researches.
•A developed SD-GCN model for accuracy pavement crack extraction from point clouds.•Feature maps and saliency matrices construction for high-level feature encodings.•Multi-scale graph-based efficient ...network for inherent point feature and edge feature representations.•Cylinder-based dilated convolution strategy for computational efficiency improvement.
Accurate pavement crack extraction is significant for pavement routine maintenance and potential traffic disaster minimization. Due to unordered data formats, intensity distinctions, and crack shape variations from point clouds captured by mobile laser scanning (MLS) systems, many preceding rule-based approaches and learning-based approaches cannot achieve high extraction accuracy and efficiency. To tackle these problems, we develop a saliency-based dilated graph convolution network, named SD-GCN, for pavement crack extraction from MLS point clouds. This network mainly consists of four modules. First, Module I is designed to remove off-ground point clouds. Next, two feature saliency maps are constructed to leverage both height and intensity information in Module II. Then, in Module III, the inherent point features and high-level edge features in multiple local neighborhoods are further extracted using a cylinder-based dilated convolution strategy. Finally, an MLP-based net architecture is designed for crack extraction refinement in Module IV. Experimental results exhibit that the SD-GCN model delivers an average of precision, recall, and F1-score of 79.5%, 77.1%, and 78.3%, respectively, which outperforms state-of-the-art methods in terms of extraction accuracy and computational efficiency.
The mobile laser scanning (MLS) technique has attracted considerable attention for providing high-density, high-accuracy, unstructured, three-dimensional (3D) geo-referenced point-cloud coverage of ...the road environment. Recently, there has been an increasing number of applications of MLS in the detection and extraction of urban objects. This paper presents a systematic review of existing MLS related literature. This paper consists of three parts. Part 1 presents a brief overview of the state-of-the-art commercial MLS systems. Part 2 provides a detailed analysis of on-road and off-road information inventory methods, including the detection and extraction of on-road objects (e.g., road surface, road markings, driving lines, and road crack) and off-road objects (e.g., pole-like objects and power lines). Part 3 presents a refined integrated analysis of challenges and future trends. Our review shows that MLS technology is well proven in urban object detection and extraction, since the improvement of hardware and software accelerate the efficiency and accuracy of data collection and processing. When compared to other review papers focusing on MLS applications, we review the state-of-the-art road object detection and extraction methods using MLS data and discuss their performance and applicability. The main contribution of this review demonstrates that the MLS systems are suitable for supporting road asset inventory, ITS-related applications, high-definition maps, and other highly accurate localization services.
•Combing usage of QuEChERS and DLLME for the determination of neonicotinoid residues in grains.•Smart role of water as extractant in QuEChERS and as external phase in DLLME.•Extraction relay using ...MeCN after water in QuEChERS and successive play of MeCN as dispersant in DLLME.•Comprehensive consideration of the single factor test and response surface method to optimize parameters.•Good sensitivity, precision, and applicability of the developed method for real grains samples.
Monitoring neonicotinoid residues in grains is of significant interest for the proper assessment of pesticide exposure to human. The quick, easy, cheap, effective, rugged, and safe extraction method combined with dispersive liquid-liquid micro-extraction (QuEChERS-DLLME) was developed for extracting, purifying, and concentrating seven common neonicotinoid pesticides from the grains (rice, millet, and maize). Water and acetonitrile were used in tandem as extractants in QuEChERS, while water, acetonitrile, and trichloromethane in DLLME acted as the external phase, dispersant, and extractant, respectively. Comprehensive consideration of the single factor test and response surface method to optimize parameters including type and volume of extractants and dispersant. The evaluation showed that the QuEChERS-DLLME method held excellent linearity (R2 > 0.99). The limits of quantitation ranged from 0.003 to 0.08 µg kg−1 for the seven insecticides. The recoveries were in the range of 62–118%, and good reproducibility was obtained with a relative standard deviation below 15%.
Disinfection byproducts (DBPs) remains an ongoing issue because of their widespread occurrence and toxicity. Various organic substances in Algogenic organic matter (AOM) can produce DBPs in the ...chlorination process. To provide specific suggestions for the targeted removal of DBP precursors in AOM, the main biochemical components in AOM were qualitatively and quantitatively analyzed. An accurate model for predicting the DBP formation potentials (DBPFPs) of AOM was herein developed based on the dissolved organic carbon of the five main biochemical components in AOM and the DBPFPs of their corresponding surrogates. The contributions of each biochemical component to the three DBP species were evaluated, and the key components were identified. The results showed that lipids, proteins, carbohydrates, humic acid-like substances, and fulvic acid-like substances were the main biochemical components in AOM. Thereof, proteins (71.2 ± 2.1%) and carbohydrates (53.1 ± 2.1%) were the major contributor to the carbon content in intracellular organic matter and extracellular organic matter, respectively. The contribution results of biochemical components to the formation of DBPs showed that proteins were the key contributor to DBPs, suggesting that the targeted removal of proteins before the chlorination process would effectively reduce DBPs from AOM.
Display omitted
•Carbon contents of biochemical components in AOM were first determined.•Main biochemical components in AOM were qualitatively determined.•Proteins and carbohydrates were the major biochemical components in AOM.•Proteins in AOM were the key contributor to DBPs during chlorination of AOM.
•Application to the feature descriptions for multimodality remote sensing image matching.•Contextual self-attention module to add global semantic dependency information.•Cross-fusion module to the ...interaction of information between reference and sensed images.•Mapping the similarity matching into a classification task by the designed loss function.
Effective feature description for cross-modal remote sensing matching is challenging due to the complex geometric and radiometric differences between multimodal images. Currently, Siamese or pseudo-Siamese networks directly describe features from multimodal remote sensing images at the fully connected layer, however, the similarity of cross-modal features during feature extraction is barely considered. Therefore, we construct a cross-modal feature description matching network (CM-Net) for remote sensing image matching in this paper. First, a contextual self-attention module is proposed to add semantic global dependency information using candidate and non-candidate keypoint patches. Then, a cross-fusion module is designed to obtain cross-modal feature descriptions through information interaction. Finally, a similarity matching loss function is presented to optimize discriminative feature representations, converting a matching task into a classification task. The proposed CM-Net model is evaluated by qualitative and quantitative experiments on four multimodal image datasets, which achieves the average Matching score (M.S.), Mean Matching Accuracy (MMA), and average Root-mean-square error (aRMSE) of 0.781, 0.275, and 1.726, respectively. The comparative study demonstrates the superior performance of the proposed CM-Net for the remote sensing image matching.
•An augmented ConvLSTM model was proposed for ocean Rrs 7-day-ahead prediction.•An attention-augmented convolution was used to extract spatiotemporal features.•The proposed model achieves a ...satisfactory performance at 443, 488 and 555 nm bands.•The proposed method outperformed the CNN, LSTM, CNN-LSTM, and ConvLSTM models.
Remote sensing reflectance (Rrs) is an essential parameter in ocean color remote sensing and a fundamental input for the estimation of ocean color elements. Predicting Rrs has the potential to enable simultaneous prediction of multiple marine environmental parameters, facilitating multi-perspective analysis of marine environmental changes. This paper proposes a spatiotemporal attention-augmented ConvLSTM-based model for ocean Rrs prediction. The developed model can predict Rrs for up to seven days by simultaneously learning spatiotemporal features from time series Rrs and auxiliary environmental variables. According to the experiments, the proposed model achieves optimal performances on Rrs predictions at 443, 488, and 555 nm, with Root Mean Squared Error (RMSE) and Mean Absolute Percentage Error (MAPE) for the first four prediction days less than 5.6*10-4 sr-1 and 8.6 %, respectively, which are better than the performance of the convolutional neural network (CNN), the LSTM, CNN-LSTM, and the ConvLSTM. The spatial and temporal variations of Rrs are also compared to evaluate the effectiveness of the model, presenting a consistent spatiotemporal pattern between predicted and observed Rrs. We also found that integrating sea surface temperature (SST), photosynthetically available radiation (PAR), and aerosol optical thickness at 869 nm (AOT869) into the model can improve the prediction accuracy in various degrees. This work suggests the proposed deep learning model can predict Rrs for 7 days with a convincing performance, providing critical data and technical support for ocean-related applications, such as algae bloom monitoring.
•Comparison of 12 loss functions for road segmentation based on deep learning.•Performances of the deep learning model differs between the loss functions.•Consistent differences in model performances ...across 2 remote sensing road datasets.
Road extraction from remote sensing imagery is a fundamental task in the field of image semantic segmentation. For this goal, numerous supervised deep learning techniques have been created, along with the employment of various loss functions that play a crucial role in determining the performances of supervised learning models. However, there is a lack of comprehensive analysis of the performance differences between the loss functions for road segmentation in remote sensing imagery. Therefore, this study conducts a comparative study of 12 well-known loss functions used widely in the field of image segmentation by training and evaluating the representative D-LinkNet network for road segmentation tasks with two publicly available remote sensing road datasets consisting of very high-resolution aerial and satellite images. The results show that different loss functions could lead to very different outcomes using the D-LinkNet, with varying focuses such as on overall model performances, precision, or recall. By dividing the loss functions into the distribution-based, region-based, and compound ones, we found that the region-based loss function type led to generally better model performances than the distribution-based one in terms of F1, IoU, and the road segmentation maps, with the compound loss function type being comparable to the region-based one. This paper eventually tries to offer suggestions for choosing the loss function that best suits the purposes of road segmentation-related studies.
•A new CNN based instance segmentation method.•The first study for instance segmentation-based oil well sites identification and extraction.•A complete framework including data fusion, object ...detection and extraction, and postprocessing for oil well sites for the task.
Fine-scale land disturbances due to mining development modify the land surface cover and have cumulative detrimental impacts on the environment. Understanding the distribution of fine-scale land disturbances related to mining activities, such as oil well sites, in mining regions is of vital importance to sustainable mining development. For efficient mapping, automated identification and extraction of the oil well sites using high-resolution satellite images are required. In this work, we proposed the Oil Well Site extraction (OWS) Mask R-CNN based on the original Mask R-CNN (Region-based Convolutional Neural Networks), to accurately extract well sites using multi-sensor remote sensing images. For improvement of mapping efficiency, two modifications were made to Mask R-CNN: (1) replacing the backbone of Mask R-CNN with D-LinkNet, and (2) adding a semantic segmentation branch to Mask R-CNN to force the whole network to focus on the relationship between line objects and oil well sites. As imagery data were from multiple sensors (RapidEye 2/3 and WorldView 3), a pre-trained Residual Channel Attention Network (RCAN) was applied to super-resolve the images with different resolutions. Several key spatial features, such as nearby roads and area size, have also been used in the oil well site mapping process. The experimental results indicate that our OWS Mask R-CNN considerably improves the average precision (AP) and the F1 score of Mask R-CNN from 51.26% and 25.7% to 60.93% and 61.59%, respectively.
•A novel Transformer-based deep neural network was proposed for pavement crack detection.•Transformer modules extracted global contextual information for long-range dependency modeling.•A local ...enhancement module was designed to compensate for fine-grained local features.•A manually annotated pavement crack dataset was built for high-resolution CCD image-based pavement crack detection.
Precisely identifying pavement cracks from charge-coupled devices (CCDs) captured high-resolution images faces many challenges. Even though convolutional neural networks (CNNs) have achieved impressive performance in this task, the stacked convolutional layers fail to extract long-range contextual features and impose high computational costs. Therefore, we propose a locally enhanced Transformer network (LETNet) to completely and efficiently detect pavement cracks. In the LETNet, Transformer is employed to model long-range dependencies. By designing a convolution stem and a local enhancement module, both low-level and high-level local features can be compensated. To take advantage of these rich features, a skip connection strategy and an efficient upsampling module is built to restore detailed information. In addition, a defect rectification module is further developed to reinforce the network for hard sample recognition. The quantitative comparison demonstrates that the proposed LETNet outperformed four advanced deep learning-based models with respect to both efficiency and effectiveness. Specifically, the average precision, recall, ODS, IoU, and frame per second (FPS) of the LETNet on three testing datasets are approximately 93.04%, 92.85%, 92.94%, 94.07%, and 30.80FPS, respectively. We also built a comprehensive pavement crack dataset containing 156 high-resolution manually annotated CCD images and made it publicly available on Zenodo.
PDF
